Using pharmacological manipulations to study the role of dopamine in human reward functioning: A review of studies in healthy adults

Abstract

Dopamine (DA) plays a key role in reward processing and is implicated in psychological disorders such as depression, substance use, and schizophrenia. The role of DA in reward processing is an area of highly active research. One approach to this question is drug challenge studies with drugs known to alter DA function. These studies provide good experimental control and can be performed in parallel in laboratory animals and humans. This review aimed to summarize results of studies using pharmacological manipulations of DA in healthy adults. ‘Reward’ is a complex process, so we separated ‘phases’ of reward, including anticipation, evaluation of cost and benefits of upcoming reward, execution of actions to obtain reward, pleasure in response to receiving a reward, and reward learning. Results indicated that i) DAergic drugs have different effects on different phases of reward; ii) the relationship between DA and reward functioning appears unlikely to be linear; iii) our ability to detect the effects of DAergic drugs varies depending on whether subjective, behavioral, imaging measures are used.

I. Introduction

A. Rationale for review

The reward system is key to adaptive functioning, as it helps us identify, respond to, and remember stimuli, such as food, that aid in survival. Dopamine (DA) is thought to play a critical role in the processing of rewards. Understanding the role of DA in reward-related functioning is clinically important, as hypo- and hyper-activity of the DA system is thought to contribute to reward-related pathologies in addiction, depression, and schizophrenia (Barch et al., 2014; Davis et al., 1991; Garfield et al., 2013; Leyton, 2010; Treadway and Zald, 2011a; Volkow et al., 2011, 2007). Much of our understanding of DA function is derived from studies with animal models, which can test highly complex brain circuits and receptor mechanisms involved in motivated behaviors in detail, with strong experimental control (Bamford et al., 2018; Berke, 2018a; Schultz, 2016a; Volkow et al., 2017). Although these studies have unquestionably advanced our understanding of how DA is involved in behavior, limitations in translating findings from animals to humans make it critical to confirm the role of DA in reward processes in humans (Stephens et al., 2011). Clinical studies have investigated reward functioning in patients with psychiatric disorders associated with DA dysfunction, such as depression and addiction (Hamilton et al., 2012; McGuire et al., 2008; Volkow et al., 2011). However, these studies provide only indirect evidence for the role of DA, because our understanding of the pathophysiology of these disorders is incomplete, and because it is difficult to rule out confounding factors related to the natural history of the disorder. Thus, pharmacological challenge studies with agents that increase or decrease DA function offer a valuable translational bridge, as they can be conducted in animals, healthy humans, and psychiatric patients.

In particular, drug challenge studies with healthy volunteers may provide critical information about the DA-related processes that mediate and maintain reward-motivated behavior. Drug challenge studies provide good experimental control, often use behavioral tasks with high translational fidelity to the animal literature, and can be combined with imaging to provide at least some degree of neuroanatomical specificity. Studying healthy adults allows investigators to examine the effect of DA manipulations in individuals whose responses are not confounded by co-existing psychiatric disorders or extensive exposure to drugs. Thus, in this paper, we focus particularly on what can be learned about reward functioning in humans from pharmacological challenge studies with dopaminergic drugs, conducted in healthy volunteers. The goal of the review is to summarize key conclusions that can be drawn from these studies about how DA mediates reward functioning in humans.

B. Scope and organization of review

i. Defining “reward”.

The term “reward” is used in many ways, from experimental procedures and events (e.g., the food reward was delivered), to purported brain processes (e.g., neural response to reward). The brain processes involved in reward are extremely complex, including memory, sensory and motor processing, motivation, and decision-making circuits. In this review we separate reward processing into five behavioral components, corresponding to five phases of processing a single reward, as derived from theoretical perspectives (Assadi et al., 2009; Berridge and Kringelbach, 2015; Berridge and Robinson, 1998; Cannon and Bseikri, 2004; Niv et al., 2007; Salamone, 2006; Schultz, 2016a; Schultz et al., 1997; Zald and Treadway, 2017). These phases are: 1) “Anticipation” - hedonic impact of anticipating future reward, 2) “Evaluation” – evaluation of the costs and benefits of an upcoming reward, 3) “Execution” – execution of behavior to obtain reward, 4) “Pleasure” - hedonic responses to receipt of a reward, and 5) “Learning” - learning from rewarding outcomes (see Figure 1). We realize this taxonomy may not perfectly segment reward-related processing and that there is likely overlap between phases. However, we felt that grouping by behavioral components was necessary to 1) directly compare studies using similar measures and 2) make the large literature manageable. Further, studies in animals indicate that these different behavioral components engage different neuroanatomical processes (Assadi et al., 2009; Berridge and Kringelbach, 2015; Berridge and Robinson, 1998; Cannon and Bseikri, 2004; Niv et al., 2007; Salamone, 2006; Schultz, 2016a; Schultz et al., 1997; Zald and Treadway, 2017). Within each phase, we further group studies based on the type of measure employed, reviewing self-report, behavioral, and imaging findings in separate sections. Finally, within measurement types, we group findings according to whether the drug administered was intended to augment or reduce DA functioning. Studies with multiple outcomes or drugs thus appear in multiple sections. The studies reviewed reflect an extensive search of key terms related to reward (e.g., anticipation, evaluation, pleasure, etc.), DA function (e.g., DA agonist, antagonist, etc.) and specific dopaminergic agents. Only studies performed on healthy adults using a pharmacological manipulation of DA were included, totaling over 100 articles included in the final review.

Figure 1.

Organizational taxonomy of reward-related processes

To provide context for interpreting study findings, we first briefly overview DA receptor types and signaling dynamics, and typical pharmacological manipulations used to study the role of DA in reward in humans. We then devote one section to each reward phase.

C. Brief overview of relevant dopaminergic neuropharmacology

i. DA receptor types

Different DA receptors have been implicated in different components of reward-related behavior in animal models. In humans, even though some drugs vary in their affinities for DA receptor types, it is difficult to delineate the role of receptor subtypes in observed behaviors. Nevertheless, because some human drug challenge studies have interpreted patterns of results based on receptor subtypes, we briefly review these subtypes to help the reader evaluate these explanations.

Five different DA receptor types have been identified (D1–D5), but in general, the focus is on two subtypes: D1-like (D1 and D5) and D2-like (D2–D4). Subtypes differ in their affinities, G-protein coupling, which intracellular pathways they stimulate, and their presumed functions (Beaulieu and Gainetdinov, 2011; Tritsch and Sabatini, 2012). D2-like receptors are present both pre- and post-synaptically, while D1-like are mostly post-synaptic (Chen et al., 2005; Leriche et al., 2008). Pre-synaptic D2-like autoreceptors have 10× to 100× greater affinity than D1-types, and thus can be activated in low-DA states, while D1 receptors require higher concentrations of DA (Cooper et al., 2003; Grace, 1991). D1 receptors are widespread in the brain and abundant in the striatum, NAcc, olfactory tubercle, and prefrontal cortex, whereas D2 receptors are more abundant in the striatum (Camps et al., 1990; Lidow et al., 1991). These differences become important when considered in context of DA signaling dynamics reviewed in the next section.

ii. DA signaling dynamics

Variations in DA signaling dynamics and levels, which can be observed directly in preclinical studies, have long been important to theories of reward-related functions. These include different modes of DA release, and non-linear responses to stimulation seen in dynamic systems.

A long-standing theory proposes that two major types of DA signaling exist in the ventral tegmental area (VTA) where DA neurons originate, each with different time courses. This theory distinguishes between phasic DA release -- stereotyped, transient bursts of activation that are brief, but release large DA concentrations sufficient to activate post-synaptic, lower affinity D1 receptors (Grace, 2000, 1991; Schultz, 2013) – and tonic DA release -- low levels of single spike firing in DA neurons which diffuse DA out of the synapse, contributing to extracellular DA, and primarily activating more sensitive D2 presynaptic autoreceptors (Floresco et al., 2003; Grace, 2000, 1991). These two types of DA signaling were hypothesized to be relevant to different reward functions, such that phasic DA was proposed to underpin reward learning, while tonic DA was thought to be involved in effort exerted to obtain a reward (Niv et al., 2007; Salamone and Correa, 2012; Schultz et al., 2017). Finally, phasic and tonic DA were thought to be related through homeostatic mechanisms, with higher levels of tonic DA thought to decrease phasic DA transmission through autoreceptors (Grace, 2000, 1991). This theory has strongly influenced the human literature, from selection of DA manipulations to interpretation of results, so we describe it here and will reference it elsewhere in the paper when it has been used by investigators to explain results. However, it is important to note that the idea of a slow, tonic DA signal has been challenged. Newer results from the animal literature suggest instead a distinction between “burst” DA releases originating in the VTA and traveling “long distance” to the NAcc and elsewhere, still thought to be critically involved in the acquisition of new learning, and “local” DA release from neurons in in the NAcc, which is thought to influence motivation and effort (Berke, 2018b; Mohebi et al., 2019). Both of these processes are now thought to operate on fast time scales, and it is unclear whether a homeostatic relationship is expected.

Responses to DA-related drugs may also be non-linear, and interact with baseline levels of DA. In particular, the “inverted-U” hypothesis suggests there is an optimal level of DA for various reward functions, and that increases beyond this level will impair function (Vaillancourt et al., 2013). Thus, increasing DA could either enhance or impair reward functioning, depending both on whether the organism’s baseline DA levels are low or high, and on the optimal level of DA for that particular function (Cools, 2019; Cools and D’Esposito, 2011; Kroener et al., 2009). However, there is also an alternate possibility, which could be characterized as, “the rich get richer, while the poor get poorer”. In this hypothesis, individuals with good baseline DA receptor capacity might benefit more from DA augmentation (Swart et al., 2017), and conversely, the effects of DA reduction might only be evident in individuals with poor baseline functioning (Kelm and Boettiger, 2013). Under either of these theories, individual differences in baseline DA could moderate the effects of DA challenges, so we note when baseline differences are explored as possible moderators.

Potential baseline differences in DA have been measured in several ways in the literature we will review. PET imaging is the “gold standard”, but the expense of this approach means that in the reviewed articles, baseline functioning has more often been imputed from genetic polymorphisms or behavioral proxy measures. In genetics, the COMT Val148Met polymorphism has been most commonly examined. Homozygote carriers of the val allele are thought to have less prefrontal DA at baseline (Nicholson, 2002); however, some research suggests that the val allele may also produce higher tonic DA in the midbrain (Corral-Frías et al., 2016), complicating interpretations of interactions between COMT genotype and drug challenges, as systemic challenges are not regionally specific. Behavioral measures that have been used as proxies of baseline DA levels include working memory (Cools et al., 2008; Landau et al., 2009), impulsivity (Cools et al., 2008), and extraversion (Wacker and Gatt, 2010).

D. Pharmacological challenge strategies

The drugs used to alter DA function in humans fall into two broad classes: those intended to increase DA function and those intended to reduce it. Drugs used to augment DA include receptor agonists (e.g. bromocriptine), re-uptake blockers (e.g. d-amphetamine), and precursors (e.g. levodopa; L-DOPA), and drugs that impair metabolism of DA (e.g. tolcapone). Drugs used to reduce DA function include receptor antagonists such as haloperidol, or the acute phenylalanine/tyrosine depletion procedure (APTD), which limits availability of DA precursor amino acids (Moja et al. 1996; Leyton et al. 2000). NMDA receptor antagonists (e.g., memantine) are also thought to decrease “phasic” or “burst” DA transmission (Jocham et al., 2014). “Low doses” of D2 agonists have also been used as a DA reduction strategy, based on the idea that activating more sensitive presynaptic autoreceptors will decrease phasic DA transmission (e.g., Pizzagalli et al. 2008a). Conversely, “low doses” of D2 antagonists have been used to increase DA, based on their presumed ability to increase phasic DA (e.g., Mueller et al., 2014). However, there is no clear agreement in the field on what constitutes a “low dose”, or when autoreceptor vs. post-synaptic effects dominate. In the absence of a pharmacologically-based cut-point, for purposes of this review, we classify studies as augmentation or reduction based on the authors’ stated hypothesis about how that drug and dose should affect DA. Occasionally this results in studies using the same drug and dose with opposing purposes. To address this, we highlight when a “low dose” strategy is being used, and note drug and dose for every study, so that readers may take this into account.

II. Anticipation of Reward

A. Anticipation of reward - Definition

We label the first phase of reward processing, as outlined in Figure 1, “Anticipation”. We define anticipation as the hedonic impact of expecting a reward, e.g. subjective excitement about future or possible reward. Anticipation starts as soon as a future reward is perceived and continues up until the reward has been received. Anticipation is also clearly related to evaluation of benefits in reward-related decision-making, but measures involving explicit decision-making are discussed in “Evaluation”. Similarly, anticipation is intimately involved in predicting the value of a reward in reward-related learning, but studies that measure changes in behavior over repeated trials are discussed in “Learning”. Here we focus on studies that simply examine anticipation without measuring impacts on decision-making or future behavior.

B. Anticipation of reward – Predictions based on putative neurobiology

Neurobiological studies suggest that DA manipulations should affect anticipation in humans. In laboratory animals, the areas associated most strongly with reward anticipation are the amygdala, VTA, NAcc, and ventral pallidum (Berridge and Robinson, 2003). DA increases in the VTA and NAcc of rats during presentation of a conditioned stimulus associated with reward, prior to receipt of the reward (Schultz, 2016b). fMRI studies in humans show that anticipation of reward activates subcortical limbic and prefrontal brain regions, including the NAcc, thalamus, striatum, prefrontal cortex, and ACC (Oldham et al., 2018; Schott et al., 2008; Schultz, 1998). Although fMRI is not pharmacologically specific, these areas are in known dopaminergic pathways.

Behavioral studies in animals also support the idea that DA is involved in anticipation. Although anticipation is more difficult to measure in animals, rats emit ultra-sonic vocalizations when rewards are expected that are thought to indicate anticipation (Knutson et al., 1998). Systemic administration of dopamine antagonists reduces these calls in male rats anticipating sexual contact with a female, consistent with the idea that systemic manipulations of DA should alter anticipation (Ciucci et al., 2007).

Based on the putative neurobiology of reward anticipation, we hypothesize that pharmacological challenges that increase DA in humans will generally increase anticipation of reward, while reduction of DA will decrease anticipation.

C. Anticipation of reward - Measures

Anticipation of reward in humans has been assessed using self-report, behavioral measures and neuroimaging. Self-reports have been administered before the reward has been received, retrospectively after the reward is received, or when imagining future hypothetical rewards (Sharot et al., 2009), and typically consist of simple happiness or excitement ratings. One study examined a behavioral measure, skin conductance, in response to anticipation of reward. However, the most common method is neuroimaging, with most studies using fMRI during cues signaling the possibility of reward.

One task, the Monetary Incentive Delay (MID) task (Knutson et al., 2000), has been most often used for imaging anticipation, so we describe it in more detail. The MID requires participants to a) view a cue that corresponds to a reward value; b) view a fixation cross during a delay period; c) respond quickly to a target by pressing a button; d) view a fixation cross during a delay period; and e) get feedback on whether they received a reward. Receipt of reward is based on the speed of the participants’ response to the target, with the required speed titrated to produce a certain percentage of rewarded trials. Depending on the study, anticipation may be measured during the cue, during the delay before the participant completes the action to receive a reward, or after the individual completes the task but before feedback on the outcome. In this review, reaction time in response to the cue is considered to represent “Execution”, and is discussed in that section.

Studies on anticipation have used a range of reward types including real and hypothetical monetary rewards and pleasant/unpleasant tastes, so reward type will be noted as a possible moderator.

D. Anticipation of reward - Summary of Studies

i. Anticipation of reward – Self-report measures

Three studies have tested the effect of drugs intended to increase DA on self-reported anticipation. Only one was consistent with our hypothesis that increasing DA would increase anticipation. In this study, L-DOPA (100mg) enhanced participants’ estimates of the pleasure they would derive from hypothetical future vacations (Sharot, Shiner, Brown, Fan, & Dolan, 2009). However, in another study that used “real time” ratings before actual events, the DA agonist bupropion (150 mg/day for 7 days, experiment occurred on day 7) had no effect. In this study, participants viewed pictures of either chocolate or moldy food, which signaled delivery of either a rewarding pleasant taste (chocolate) or a punishing unpleasant taste (mixture of chocolate and beetroot juice), respectively. Bupropion had no effect on reports of how much participants “wanted” the pleasant taste, measured just prior to taste delivery (Dean et al., 2016). A final study produced results directly contrary to our hypothesis. In this study, participants either received amphetamine or placebo while performing the MID task, and retrospectively reported “excitement” during cues that signaled upcoming monetary reward. d-Amphetamine (.25 mg/kg), produced a non-significant trend to report less excitement while viewing a reward cue (+$5.00), and significantly increased reports of excitement while viewing loss cue (−$5.00). Thus, DA augmentation seemed to “flatten” anticipation to reward and loss (Knutson et al., 2004). In summary, drugs that increase DA function have had mixed effects on self-reported anticipation. Of note, each study used a different reward type, drug, time frame for assessment and scale, making it difficult to assess contributions of design differences.

Only one study has examined the effect of a manipulation intended to decrease DA on self-reported anticipation. Depletion of DA via APTD did not change retrospective reports of excitement during monetary reward cues on the MID (Bjork et al., 2014). This is also inconsistent with our hypothesis.

ii. Anticipation of reward – Behavioral measures

Only one study has examined the effects of a dopaminergic modulation on a behavioral response during anticipation of reward. This study examined skin conductance, a physiological measure of arousal, during the anticipation period of the MID (Ferreri et al., 2019). Increasing DA with L-DOPA (100mg) increased electrodermal responses, compared to reducing DA with risperdone (2mg; although a placebo was included, placebo comparisons were not reported). Importantly, these changes were not evident during no-reward trials, indicating they were not a physiological artifact of the drugs. This result is consistent with our hypothesis.

ii. Anticipation of reward – Neuroimaging measures

Fifteen studies have examined the effects of drugs intended to increase DA function on brain activity during reward anticipation. The most commonly reported results are for the striatum, including the ventral striatum, which encompasses the NAcc, and the dorsal striatum or caudate, so we will focus on these.

Seven studies found that drugs intended to increase DA function increased striatal activity, consistent with our hypothesis. In the Dean et al (2016) study already described above, bupropion (150mg/day for 7 days) increased BOLD activity in the caudate, pregenual ACC/ventromedial prefrontal cortex and lateral orbitofrontal cortex during anticipation of pleasant tastes (Dean et al., 2016). The drug also increased caudate activity during anticipation of unpleasant tastes, suggesting increasing DA may also increase anticipation of punishment/loss. Six other studies examined the effects of DA-increasing drugs during anticipation of monetary rewards. Methylphenidate (40mg) increased BOLD activity in the ventral striatum (Evers et al., 2017), modafinil (200mg) increased bilateral NAcc activation (Funayama et al., 2014), and bupropion (150mg) increased right Nacc activation (Ikeda et al., 2019), all during anticipation of monetary reward. The agonist bromocriptine (1.5mg) also increased activity in the right NAcc, but only in individuals with the A1A1 DRD2 genotype (Kirsch et al., 2006). This genotype is related to lower baseline levels of DA, so these findings are consistent with an inverted-U relationship in which individuals low in DA at baseline experience a greater boost in anticipatory activity from DA-increasing drugs. Finally, a paradigm designed to parallel animal studies of sensitization, a phenomenon in which repeated administration of stimulants results in increased DA release at subsequent administrations, administered d-amphetamine (20mg) or placebo every 48 hours for three sessions, and then again a final time after a two-week abstinence period. At the final administration, BOLD activity in the caudate was increased in the d-amphetamine condition during anticipation of monetary reward (O’Daly et al., 2014).

While each of the above studies examined anticipation when individuals were preparing to make a response, one additional study used a task in which participants were also asked to withhold responses to get reward. DA is thought to play a dual role in both anticipation of reward and invigoration actions taken to gain reward (this will be further discussed in the “Execution” section). Based on this dual role, Guitart-Masip and colleagues argued that increasing DA might preferentially enhance anticipation of actions taken to gain reward (“go to win” actions), but not actions withheld to gain reward (“no-go to win”), or actions taken to avoid punishment (“no-go to avoid punishment”; Guitart-Masip et al., 2012a). In an fMRI paradigm that dissociated anticipation from preparation for action by crossing go and no-go response requirements with reward and punishment, these authors found that L-DOPA (150mg) increased activity in the striatum and substantia nigra/VTA only during anticipation of “go to win” trials, not “go to avoid punishment” or “no-go to win” trials. Taken together, these seven studies suggest that DA augmenting drugs boost activity in the striatum (broadly defined) during anticipation.

Null results were seen in five other studies. Benzylpiperazine, a “designer” drug thought to have primarily DAergic effects (Curley et al., 2013), did not affect activity in the striatum during reward anticipation, although it decreased activity in inferior frontal gyrus, insula, and mid-occipital regions. However, it is not clear that this study assessed anticipation of reward per se, as it used a task in which participants did not know whether they were expecting a reward or loss (50/50 chance). Three studies using the more typical MID task showed no effect of bupropion (150ng) or atomoxetine (40mg) within the NAcc (Graf et al., 2016a; Suzuki et al., 2019), or of L-DOPA (100mg) on broader striatal activity during reward anticipation (Wittmann et al., 2015). However, one of these studies (Wittmann et al., 2015) did see an increase in striatal activity when participants anticipated monetary losses. A final study examined effects of L-DOPA (150mg) in a MID-like task with auditory, rather than visual, cues. In this study the drug did not affect responses in the striatum during anticipation, but increased responding in the left auditory cortex, left inferior frontal gyrus, and ACC (Weis et al., 2012).

Three imaging studies suggested that DA increasing drugs decrease activity in the striatum during anticipation of reward. In one study methylphenidate (0.5mg.kg) decreased activity in the caudate, left thalamus and right cerebellum during anticipation of reward in an “anticipation, reward, conflict” task (Ivanov et al., 2014). In another study d-amphetamine (0.25 mg/kg) “flattened” levels of ventral striatal activity during the MID by reducing maximal activation during anticipation of gains, but increasing activation during anticipation of losses (Knutson et al., 2004). However, despite the reduction in maximal activation during reward anticipation, d-amphetamine also prolonged striatal activity during this phase. The authors suggested this might reflect increased tonic DA activity blunting phasic DA activity via autoreceptors. Unfortunately, no future studies have examined duration of activation to further test this hypothesis. A final study used MEG to study reward function after treatment with L-DOPA (150mg). L-DOPA reduced ERF amplitude over the occipital areas, specifically to cues predicting high probability, high magnitude reward (Apitz and Bunzeck, 2014). The ERF effects were accompanied by decreased power in the beta band over the frontal areas during anticipation. ERFs during anticipation are sensitive to outcome valence (Doñamayor et al., 2011; HajiHosseini et al., 2012; Marco-Pallares et al., 2008), and beta power is thought to reflect the linking of stimulus-response associations in the dorsolateral prefrontal cortex (HajiHosseini and Holroyd, 2015; Melgari et al., 2014). Thus, these results also suggest that L-DOPA may decrease anticipation of reward in both visual and frontal areas. However, these findings are difficult to compare with fMRI studies that examined primarily striatal regions.

In sum, we hypothesized that drugs that increase DA function would enhance striatal activity during reward anticipation, and a plurality of studies supports this. Of note, although responses to punishment are not a focus of this article, findings from several studies suggest DA increasing drugs may also increase striatal responses to anticipated punishment.

Seven studies have explored the effects of DA reduction on brain activity during anticipation, and of these, five found decreases in activity, consistent with our hypothesis. Four of these studies utilized fMRI. In one, the antagonist olanzapine (5mg) decreased activity during reward anticipation in the MID task in VS, ACC, and inferior frontal cortex (Abler et al., 2007). In two other studies, APDT reduced reward-related corticostriatal activation during the anticipation period of a rewarded pattern-matching task and the MID task, particularly in the ventral striatum and NAcc (Bjork et al., 2014; Nagano-Saito et al., 2012). Another study using risperidone (0.5mg) found decreases in dorsal striatum, and orbitofrontal cortex during anticipation of reward in a task requiring application of precise amount of force to gain a reward -- although it should be noted that this study omitted a “no reward” condition (Fiore et al., 2018).

In addition to these fRMI studies, one study recorded EEG during a Go/No-Go task. The authors measured the reward-related positivity, probability-related positivity, and P300 waveforms in response to a cue that indicated a button press needed to be made to obtain reward (Schutte et al., 2020a). These ERPs occur in response to rewards or probability of upcoming reward, and thus may be DA-dependent. On each trial, participants were told they could win 0 or 5 Euros and the probability of winning given the correct button press was either 50% or 98%. The reward-related positivity and P300, but not the probability-related positivity, were attenuated by haloperidol, suggesting that DA is involved in the anticipation of rewarding events rather than estimating the probability an event will occur. This pattern was more pronounced in individuals with a higher spontaneous eye blink rate, thought to reflect higher baseline dopamine, consistent with an inverted-U relationship between DA and anticipation.

One study utilizing a-methylparatyrosine (1.5g), a competitive inhibitor of tyrosine hydroxylase, found null results in striatum, but increased BOLD activation in the left cingulate gyrus during anticipation on the MID task (da Silva Alves et al., 2011). Finally there was one contradictory result, in which “low dose” pramipexole (0.5mg) increased NAcc activity during the anticipation phase of the MID, but also weakened connectivity between the NAcc and prefrontal cortex, another key dopaminergic area (Ye et al., 2011). The authors speculated that this pattern could represent opposing effects of pramipexole on tonic and phasic DA, although this possibility cannot be directly tested in humans. Overall, studies on DA reduction generally add support to the idea that there is a relationship between DA levels and striatal activation during anticipation of reward.

E. Anticipation of reward - Summary, conclusions, recommendations

Our hypothesis, based on the putative neurobiology of anticipation, was that pharmacological challenges intended to increase DA would increase anticipation of a rewarding outcome, while conversely, decreasing DA would dampen anticipation of a rewarding outcome. fMRI findings were fairly consistent with this hypothesis, with a plurality of studies supporting the idea that increasing DA enhances striatal activity during anticipation, and reducing DA reduces striatal activity during anticipation. The single existing behavioral study also supported our hypothesis. However, these results are difficult to interpret in the presence of highly mixed self-report findings. It is certainly the case that there were few self-report studies, and most that existed were not directed at investigating self-report outcomes, and thus may not have been powered to examine these outcomes. Indeed, the sole study that primarily focused on self-report found that DA augmentation increased anticipated pleasure (Sharot et al., 2009). Thus, a primary recommendation in this area is to include more sensitive measures of subjective effects, and power studies accordingly.

One other feature of this literature is that the majority of studies of anticipation use the MID. This enables easier comparison across studies, but it also limits the conclusions that can be drawn about reward types. There are also variations among MID tasks that may affect outcomes, such as the timing of measures of “anticipation”, use of gain cues only vs. both loss and gain cues, and size of rewards. Finally, one study indicates that the action requirements of a task (i.e. taking vs. withholding an action) may critically moderate the effects of DA, a suggestion that should be replicated and examined further.

III. Evaluation of Rewards

A. Evaluation of rewards – Definition

We label the second stage of reward processing, as outlined in Figure 1, “Evaluation”. We define evaluation as the process of appraising the relative costs and benefits of a given reward, ending with the formation of a preference (Assadi et al., 2009; Bailey et al., 2016; Zald and Treadway, 2017). Evaluation is also referred to in the literature as reward-related, or cost-benefit, decision-making. As evidence indicates that different brain areas are involved when reward is weighed against costs involving time vs. risk vs. effort (Bailey et al., 2016; Winstanley and Floresco, 2016), we further divide the evaluation section into sub-sections on time, risk and effort-related decision-making.

C. Evaluation of rewards – Predictions based on putative neurobiology

Neurobiological studies suggest that DA manipulations should affect evaluation of reward in humans. Studies in laboratory animals indicate there is both a core DAergic system engaged in all evaluations of reward, and areas uniquely attuned to time, probability and effort costs. Animal studies suggest the “core system” for reward-related decision-making comprises the VTA, NAcc and basolateral amygdala (Bailey et al., 2016). However, the lateral orbitofrontal cortex, prelimbic and infralimbic areas of the medial prefrontal cortex and subthalamic nucleus appear to play more prominent roles in decisions involving delay, while the medial orbitofrontal cortex has a more critical role in decisions about risk, and effort-related decision-making appears to rely more on the ACC (Bailey et al., 2016). Imaging studies in humans converge to a certain extent with these findings in animals. For example, they similarly indicate that the ACC is more heavily implicated in effort-related decisions (Dreher, 2013; Massar et al., 2015; Prévost et al., 2010; Vassena et al., 2014). Human imaging studies also indicate that the vmPFC and VS (which encompasses the NAcc) are sensitive to probability and delay costs (Dreher, 2013; Prévost et al., 2010), although their role in weighing effort costs remains an area of active debate (Chong et al., 2017; Westbrook et al., 2019), thus raising the possibility of divergence from the “core system” identified in animals. These findings emphasize the importance of confirming animal findings in humans, and the need to test the impact of DA challenges on these different costs separately.

Consistent with the idea that the DAergic system is involved in cost-benefit evaluation, the literature in animals indicates that drugs that increase DA generally increase willingness to endure delays, risks, and effort to gain reward; while drugs that decrease DA have the opposite effect (Floresco et al., 2008; Koffarnus et al., 2011; Salamone et al., 2018, 2012, 2009; St Onge and Floresco, 2009; van Gaalen et al., 2006; Wade et al., 2000; Winstanley et al., 2003). However, consistent with findings that there are also cost-specific areas, evidence from animal studies suggests that optimal DA levels may vary across different cost-benefit calculations. For example, high doses of amphetamine decreased willingness to exert effort for reward (consistent with an inverted-U effect), but did not affect willingness to endure delay (Floresco et al., 2008).

Based on the putative neurobiology of reward evaluation, including existence of a general DAergic “core system”, our overall prediction is that pharmacological challenges that increase DA in humans will promote preferences for larger, more costly (more delayed/higher risk/more effortful) rewards, while reducing DA will promote a preference for smaller, less costly rewards. However, in recognition that there may also be cost-specific circuitry and dose-response curves, we review evidence for each type of cost-benefit decision-making separately.

B. Evaluation of rewards - Measures

Reward evaluation in humans has been assessed using self-report, behavioral, and neuroimaging measures. Self-report measures include Visual Analogue Scales (VAS) measuring motivation to perform a rewarded task (Baranski et al., 2004; Volkow et al., 2006). Behavioral measures include discounting tasks, in which individuals repeatedly select between low cost/low reward vs. higher cost/higher reward options. These tasks produce discounting rates, which indicate how quickly the reward is devalued as the cost rises; thus, “increased discounting” indicates less willingness to accept delay, risk or effort to gain the reward (Acheson and de Wit, 2008; de Wit et al., 2002; Kelm and Boettiger, 2013; Massar et al., 2015; Pine et al., 2010). There are also many tasks specific to evaluating decisions about risk vs. reward. One is the Balloon Analog Risk task, in which participants get a small reward for each “pump” added to a balloon, while each pump also increases the risk of the balloon exploding. If a balloon explodes, no reward is given (Acheson and de Wit, 2008; White et al., 2007). Gambling tasks are also commonly used to study the evaluation of risk vs. reward (Ojala et al., 2018; Rigoli et al., 2016b; Rutledge et al., 2015). There are also behavioral measures of effort-related decision-making, which require physical effort (e.g. button-pressing and hand grip strength), or cognitive effort (e.g. set-switching) to gain reward (Kurniawan et al., 2011; Lopez-Gamundi and Wardle, 2018; Reddy et al., 2015). A few behavioral tasks simultaneously manipulate multiple costs, such as the Effort Expenditure for Rewards Task (EEfRT) developed by Treadway et al. (2009), which requires participants to choose between a low effort/low reward option and a high effort/high reward option, both of which vary in probability of a reward being obtained (Treadway et al., 2009). Finally, neuroimaging of the evaluation phase has generally used fMRI to measure BOLD signals during cues that signal the time, risk or effort required to obtain rewards (Botvinick et al., 2009; Croxson et al., 2009; Guitart-Masip et al., 2011; Hauser et al., 2017; Kurniawan et al., 2013, 2010). Until now, most studies on the effects of DA manipulations on reward have used money as the reward. Whether these findings will generalize to other forms of reward remains to be determined.

D. Evaluation of rewards - Summary of studies

i. Evaluation of time vs. reward - Behavioral measures

Five studies have tested the effects of DA augmentation on decisions involving weighing reward against time costs, with conflicting findings. Two studies found decreases in delay discounting, consistent with our hypothesis that DA augmentation would make individuals more willing to endure delays to gain reward. In the first, augmenting DA using d-amphetamine (20mg) decreased delay discounting (de Wit et al. 2002b). In the second, the COMT inhibitor tolcapone (200mg) decreased delay discounting, particularly in individuals with high baseline impulsivity, which was used as a proxy of low baseline DA functioning (Kayser et al. 2012). This would be consistent with an inverted-U relationship between DA and delay discounting.

However, there were also three null findings. In the study that found effects of d-amphetamine on a traditional delay discounting task (which measures the effect of delays up to many months long), d-amphetamine did not affect discounting in an “experiential” task in which participants experienced short delays and small rewards in the lab (de Wit et al. 2002b). An additional study also found no significant effects of either d-amphetamine (20mg) or bupropion (150mg or 300mg) on the traditional delay discounting task (Acheson & de Wit, 2008). This null study tested the same dose of d-amphetamine as the de Wit et al. 2002 study, in a similar number of participants, providing directly contradictory information. Another study found no effect of increasing doses of the novel D1 agonist PF-06412562 (6mg, 15mg, or 30mg) on delay discounting (Soutschek et al., 2020). Finally, a single study found effects opposite to our hypothesis -- increased delay discounting when augmenting DA using L-DOPA (150mg) (Pine et al., 2010). Thus, taken together the literature suggests highly inconsistent effects of DA augmentation on delay discounting, without a clear pattern of moderation by other variables.

Six studies have tested effects of reducing DA on delay discounting, and of these, none found overall increased delay discounting consistent with our prediction. The one study that reported increased discounting in some participants used APTD, and studied individuals who varied in COMT genotype. In that study, the depletion procedure increased discounting only in participants with the val/val COMT genotype (Kelm and Boettiger, 2013). This drug/genotype interaction suggests the importance of genotype, but is otherwise difficult to interpret. The results could be consistent with a “poor get poorer” phenomenon, in which effects of DA depletion are only evident in val/val individuals, who are thought to have with lower prefrontal DA functioning at baseline. However, as noted in the introduction, the val/val COMT genotype may also increase DAergic tone in the midbrain (Corral-Frías et al., 2016), in which case this finding could also be consistent with an inverted-U. Given the small sample size and complexity of interpretation, replication and extension with imaging is needed to fully interpret this result.

Two studies failed to detect any effects of DA reducing drugs: Neither the DA receptor antagonist haloperidol (1.5 mg) (Pine et al., 2010) nor “low dose” pramipexole (0.25 and 0.5mg; Hamidovic, Kang, & de Wit, 2008) altered delay discounting.

Finally, three studies found that DA antagonists actually decreased delay discounting, contradictory to our prediction. The D2 antagonist, metoclopramide (10mg) decreased delay discounting. Although in this study delay was confounded with probability -- delayed rewards were also always more certain than immediate rewards -- computational modeling indicated that metoclopramide primarily reduced the impact of delay on decisions (Arrondo et al., 2015). The D2/D3 antagonist amisulpride (400mg) also decreased delay discounting overall in one study (Weber et al., 2016), and in another decreased delay discounting in individuals with lower BMI (and thus presumably higher amisulpride levels) (Soutschek et al., 2017). In summary, the plurality of studies were inconsistent with our general hypothesis that decreasing DA should lead individuals to prefer less costly, sooner rewards. Indeed in some cases, particularly when D2 antagonists were used, DA reduction appeared to make people more willing to wait for reward.

ii. Evaluation of time vs. reward - Neuroimaging measures

Two studies have used neuroimaging to study the effects of DA augmentation on delay discounting. In one study, the researchers first used computational modeling to identify brain areas associated with delay discounting parameters, including the “discounting factor”, i.e. the extent to which delay devalued rewards, and the “net utility”, i.e. the overall value of an option after discounting. Regions associated with the discounting factor included the striatum, insula, subgenual cingulate, and lateral orbitofrontal cortices, while net utility was related to activity in caudate, insula, and lateral inferior frontal regions. Augmenting DA with L-DOPA (150mg) increased activity in areas associated with the discounting factor, while regions associated with net utility of an option became less active. These imaging results were consistent with behavioral findings from the same study showing that L-DOPA increased delay discounting (Pine et al., 2010). Another study compared regions within a pre-determined frontostriatal mask, and augmented DA with the COMT-inhibitor tolcapone (Kayser et al., 2012). This study found that tolcapone increased activity in similar regions as L-DOPA, including striatum, pregenual cingulate and anterior insula, dorsolateral and medial frontal cortex. However, these increases took place in the context of opposite behavioral findings – a decrease in delay discounting. Taken together, although brain activity changes were fairly consistent, they are difficult to interpret in the presence of opposite behavioral results. The authors of the tolcapone study suggested that because COMT preferentially regulates DA tone in the prefrontal cortex, effects of tolcapone might be specific to the mesocortical DA system. Although regional specificity could explain different behavioral results for tolcapone vs. L-DOPA, this hypothesis is difficult to reconcile with the similar regions identified by imaging in both studies. A direct comparison of different augmentation strategies in the same paradigm and participants is needed to resolve these questions.

Regarding reductions in DA, two studies have examined delay-discounting in conjunction with imaging. The first tested the effect of haloperidol (1.5 mg) on the same brain areas associated with the “discounting factor”, and “net utility” as described above. This study found no effect on BOLD (Pine et al., 2010), consistent with null behavioral results for haloperidol in the same study. Another study examined the effect of reducing DA using the antagonist metoclopramide, using a whole brain approach (Arrondo et al., 2015). This study showed reduced activity in the post-central gyrus in the context of a decrease in delay discounting. Thus, although D2 antagonism has generally produced behavioral reductions in discounting, these behavioral results have not been accompanied by similarly consistent imaging findings.

iii. Evaluation of risk vs. reward - Behavioral measures

Twelve studies have used behavioral measures to investigate the role of DA in evaluation of probability costs vs. rewards. These broadly group into null studies using probability discounting and the Balloon Analog Risk Task (BART), vs. gambling tasks, which show more complex findings.

Examining probability discounting and the BART, there have been three studies of DA augmentation, all with null results. Two studies examining probability discounting (Acheson and de Wit, 2008; de Wit et al., 2002) with two different DA augmentation strategies (d-amphetamine 10mg and 20mg, bupropion 150mg and 300mg) found no effects. One of those studies also investigated the BART with similar null results (d-amphetamine 20mg, bupropion 150mg and 300mg) (Acheson and de Wit 2008b). Another study found null effects of d-amphetamine (20mg) on the BART overall, but these were moderated by both gender and trait reward sensitivity, such that d-amphetamine increased risk-taking in males with high reward sensitivity, decreased risk-taking in males with low reward sensitivity, and did not affect risk-taking in women (White et al., 2007). If reward sensitivity is interpreted as a proxy of baseline DA, these results could be partly consistent with a “rich get richer” phenomenon, but the further interaction with gender is not straightforwardly explainable.

In contrast to these generally null results for discounting paradigms and the BART, gambling tasks show more complex effects of DA increasing drugs. Seven studies have examined gambling tasks with DA increasing drugs. Two found increases in risk-taking when enhancing DA using L-DOPA (both 150mg), which is consistent with our overall hypothesis, although in one study this effect was only evident in individuals with low body weight (Rigoli et al., 2016b; Rutledge et al., 2015). Two found null results, such that methylphenidate (40mg) and L-DOPA (100mg) did not change risk-taking (Evers et al., 2017; Symmonds et al., 2013). Contrary to our hypothesis, one study found that increasing doses of the selective D1 receptor agonist PF-06412562 (6, 15, or 30 mg) reduced preferences for high-risk gambles (Soutschek et al., 2020).

The final two studies found more complex effects not accounted for by our hypothesis. In one study, augmenting DA using methylphenidate (20mg) increased risky “double or nothing” choices for high stakes bets, but decreased them for low stakes bets (Campbell-Meiklejohn et al., 2012). The authors suggest this pattern represents insensitivity to loss, which could reduce “loss chasing” for smaller losses, but also reduce the perceived risk of higher value “double or nothing” gambles. The second study assessed the effects of cabergoline (1.5mg) on an “experimental gamble” task, in which participants selected repeatedly between a control gamble, with 50:50 odds of 10 point gain vs. a 10 point loss and an “experimental gamble”, with odds ranging from 40–60% and gains and losses ranging from 30 to 70 points. It is difficult to interpret selection of the experimental option as risk taking, given that at times the experimental option had a higher probability of reward. However, cabergoline did decrease the extent to which losses impacted subsequent choice of the experimental option. This is broadly consistent with the prior loss chasing in indicating that increasing DA decreases the impact of loss. Cabergoline also increased selection of higher probability gambles, while decreasing selection of lower probability gambles (Norbury et al., 2013). This is consistent with a reduction in “probability distortion”, a common psychological error in which individuals generally underestimate the likelihood of high probability events, and overestimate the likelihood of low probability events.

In summary, the effects of augmenting DA on probability costs appear to depend heavily on type of task and parameters examined. Probability discounting paradigms and the BART show generally null results. Gambling tasks show mixed evidence. Some studies report increased risk taking, which may be driven by decreased sensitivity to potential losses. This is consistent with our general hypothesis that DA augmentation would reduce sensitivity to probability costs. However, gambling tasks also indicate that DA augmentation may increase use of probability information in decision-making, a different effect than predicted.

Seven studies have examined the impact of reductions in DA on evaluation of risk vs. reward. Consistent with null results for probability discounting tasks and DA augmentation, two studies using probability discounting paradigms with DA reduction also showed null results. The first, Arrondo et al. (2015) was discussed previously under delay discounting, as in this study delay was confounded with probability. Computational modeling of this task indicated that metoclopramide (10mg) affected delay, but not probability parameters. In the second study, “low dose” pramipexole (0.25mg and 0.50mg) did not affect probability discounting or the BART (Hamidovic et al., 2008).

Examining the five studies that tested gambling tasks and DA reduction again shows more complex picture, with similar results to DA augmentation. None of the results were consistent with our general hypothesis that DA reduction should produce risk-aversion. One study had null results, such that haloperidol (3mg) did not alter betting behavior on a slot machine game (Zack and Poulos, 2007). The other four showed either results directly contradictory to our hypothesis, or more complex effects. One study using “low dose” pramipexole (0.5mg) found increased risk taking, particularly when the previous trial resulted in a large gain -- a situation that usually elicits conservative decision-making (Riba et al., 2008). A further study with “low dose” pramipexole (0.5mg) found that pramipexole increased risky “double or nothing” choices for high stakes bets, but decreased them for low stakes bets (Campbell-Meiklejohn et al., 2011). This pattern was similar to results for methylphenidate on the same task, and the authors interpreted these findings as suggestive of a similar insensitivity to losses. Two additional studies indicated that reducing DA using D2/D3 antagonists also reduces “probability distortion”, similar to results found for the agonist cabergoline. In one study, reducing DA using amisulpride (400mg) increased general preference for risky options in a gambling task, while also decreasing probability distortion (Burke et al., 2018). Another study using a similar antagonist, dose and modeling parameters (sulpiride, 400 mg) did not find alterations in risk preference overall, but did find similar reductions in probability distortion (Ojala et al., 2018). In summary, reducing DA does not appear to consistently decrease risk taking. Instead, there is evidence for a potential increase, and for more subtle effects on impact of loss and use of probability information. The similarity of these findings to results for DA augmentation may be explainable in two ways. First, it is possible that the either the putative augmentation or reduction strategies described above are actually engaging pre-synaptic mechanisms, producing similar results. Second, it may be the result of an inverted-U relationship between DA and these outcomes. Either possibility strongly indicates the need for more dose-response studies in humans.

iv. Evaluation of effort vs. reward – Self-report measures

One study has tested the effect of augmenting DA on self-reported willingness to exert effort for a reward. In this study, methylphenidate (20 mg) increased self-reported motivation to complete a rewarded cognitive effort task (Volkow et al., 2004), consistent with our hypothesis.

v. Evaluation of effort vs. reward – Behavioral measures

Four out of the five studies examining the effect of DA enhancement on behavioral measures of evaluation of effort were consistent with our hypothesis that increasing DA should increase willingness to exert effort for reward. Increasing DA using d-amphetamine (20 mg) increased preference for high effort/high reward choices on the EEfRT task (Wardle et al., 2011). Of note, this increase was most evident under low probability of reward, and the high effort task also took longer to execute, so contributions of probability- and time-related decision-making cannot be ruled out. However, enhancing DA using L-DOPA (125 mg), this time in a task where time to complete the task was fixed and rewards were not probabilistic, also increased willingness to engage in physical effort for reward (Zenon et al., 2016). Similarly, augmenting DA using a D1 agonist (PF-06412562; 6mg, 15mg, 30mg) increased willingness to exert physical effort for reward (Soutschek et al., 2020). Finally, both methylphenidate (20mg) and “low dose” sulpride (400mg) increased willingness to exert cognitive effort for reward in an effort discounting paradigm (Westbrook et al., 2020). However, these effects were only evident in individuals with low baseline DA synthesis capacity (as measured by PET), suggesting an inverted-U relationship. Only one study, using L-DOPA (150mg), found no changes in willingness to exert grip-strength effort for reward (Michely et al., 2020).

Finally, four studies involving reduction of DA provide limited support for the complementary idea that depletion of DA reduces willingness to exert effort for reward. In one study women with subsyndromal seasonal affective disorder exhibited lower progressive ratio (PR) break points after APTD (Cawley et al., 2013). APTD also reduced the willingness of light smokers to work for cigarettes (Venugopalan et al., 2011) and for healthy controls to work for exercise (O’Hara et al., 2016) in a PR task. In another study (Michely et al., 2020) haloperidol (1.5mg) failed to reduce willingness to exert effort, but instead narrowed the gap in effort exerted on high vs. low reward trials, seemingly reducing sensitivity to differences in reward magnitude. In summary, the number of studies in this area is smaller, but findings suggest that modulating DA affects decisions about whether to put forth effort for reward.

vi. Evaluation of effort vs. reward – Neuroimaging measures

To our knowledge, no studies have conducted neuroimaging during evaluation of effort costs with a dopaminergic manipulation. However, PET measures of methylphenidate-induced increases in striatal DA during anticipation of a rewarded cognitive effort task were positively associated with self-report measures of motivation to engage in effort, suggesting that increases in DA relate to reward motivation (Volkow et al., 2004).

E. Evaluation of reward - Summary, conclusions, recommendations

We hypothesized that augmenting DA would generally increase willingness to endure time, risk and effort to gain reward, while reducing DA would reduce willingness to accept these costs in order to gain reward. However, we examined reward-related decisions involving tradeoffs for time, risk, and effort separately, as they are thought to have partially separable neural bases. Consistent with the idea that these domains of decision-making are distinct, we saw different patterns of results for decisions about time, risk and effort.

In the context of evaluating time vs. reward, the literature did not support our hypothesis. Augmenting DA had extremely mixed results, and a slim plurality of studies suggested that reducing DA actually had the opposite of the predicted effect, increasing the willingness of individuals to spend time waiting for rewards. Imaging studies in this area were too few and disparate in approach reach definitive conclusions.

In the context of evaluations of risk vs. reward, probability discounting procedures and the BART generally yielded null results. The apparent insensitivity of these tasks to DA manipulation may be explained by recent work in the animal literature showing that DA plays a differential role in choices involving reward omission vs. loss (Orsini et al., 2015). Discounting and BART procedures did not involve possibility of loss, while the gambling tasks generally did. Future studies in humans should attend to this aspect of probability tasks when using DA manipulations.

Gambling tasks revealed more complex effects of DA manipulation, suggesting that both DA augmentation and reduction may increase risk taking, reduce the impact of losses on choice, and ameliorate a common cognitive distortion in which individuals overestimate the probability of low chance events and underestimate the probability of high chance events. The similarity of results for augmentation and reduction may be due to either paradoxical pre-synaptic effects of DA augmentation/reduction, or represent an inverted-U relationship between DA and evaluation of probability costs. Attending to more subtle effects of DA on constructs such as loss sensitivity, along with incorporating the most recent research from the animal literature on the modulatory impact of the possibility of loss, could help clarify findings.

Finally, there was more general support for a direct relationship between DA levels and willingness to exert effort to gain reward, with the exception of two contradictory results. However, this area of work also contained the fewest studies, and is most in need of replication and extension.

IV. Execution of Actions to Attain Reward

A. Execution - Definition

We label the third stage of reward processing, as outlined in Figure 1, “Execution”. We define execution as the mobilization of resources to obtain a selected reward (Bailey et al., 2016). Execution has also been referred to as invigoration or response vigor (Kurniawan et al., 2011; Salamone et al., 2018; Treadway et al., 2009). Execution begins when a preference is generated by the evaluation phase and ends with attainment (or failure to attain) of the selected reward. Thus, in this section we cover measures taken after an explicit choice has been made, or that do not require explicit choice. There are three related theories about the role of DA in execution: 1. “Simple invigoration” – This hypothesis states that DA increases the speed or vigor of actions executed to obtain reward. Reward-related invigoration may be difficult to separate from simple locomotor effects (Bailey et al., 2016). Thus, we focus on studies that include both reward and no reward/punishment trials, and omit studies where these comparison conditions are unavailable (Wardle et al., 2011), or not analyzed (Frank and O’Reilly, 2006). 2. “Average reward rate” – This hypothesis states that DA levels represent the average rate of rewards that the organism has experienced in the environment. In this hypothesis, increased DA increases vigor by signaling that there will be lost opportunities if the organism does not quickly execute and “move on” from the current task to gain other available rewards (Niv et al., 2007). 3. “Go to win” - As described briefly in the “Anticipation” section, this theory states that DA facilitates a prepared, Pavlovian association between taking action and reward, or “go to win” actions (Guitart-Masip et al., 2011; Swart et al., 2017), as opposed to the inhibition of action for reward (i.e., “no-go to win”). Evidence related to these three theories will be covered in separate sub-sections.

B. Execution - Predictions based on putative neurobiology

Neurobiological studies suggest that DA manipulations should affect execution. (Niv et al., 2007; Salamone, 1992; Salamone and Correa, 2012). DA projections from the VTA to the NAcc appear critical to the vigor of responding for a reward (de Jong et al., 2015; Salamone et al., 2018, 2016), with some evidence suggesting that DA in the NAcc core, as opposed to NAcc shell, has the more critical role (Bailey et al., 2016). In humans, the VTA/NAcc circuit has also been implicated in execution of reward-related behaviors, with activations in the NAcc and VTA predicting degree of effort exerted (Rigoli et al., 2016a). In addition to the VTA and NAcc, the ACC, and particularly the dorsal ACC, has been implicated in sequencing, initiating and maintaining goal-directed behaviors (Assadi et al., 2009; Kurniawan et al., 2011). Indeed, it has been hypothesized that the “lower tier” of the dorsal ACC, which connects to the NAcc and hypothalamus, is particularly important in the execution phase of reward (Assadi et al., 2009). Regarding DA dynamics, the “average reward rate” hypothesis suggests that low, tonic levels of accumbens DA specifically signal the reward rate (Assadi et al., 2009; Niv et al., 2007).

Regarding the effect of DA manipulations, computational re-analyses of studies in which animals must exert effort to gain reward (Niv et al., 2007) suggest dopamine depletion primarily reduce vigor, as indicated by slower response speeds. These manipulations include specific lesions, but also systemic administration of haloperidol.

Based on the putative neurobiology of execution, we expect that augmenting DA will generally invigorate reward-related actions and increase “go” responding, whereas depleting DA will slow reward-related actions and decrease “go” responding. However, in recognition that the “simple invigoration”, “average reward rate” and “go-to-win” hypotheses presented differ somewhat in exact focus and predictions, we discuss each in separate sections.

C. Execution - Measures

Execution in humans has been assessed using self-report, behavioral and neuroimaging measures. Self-report measures consist of VAS measuring perceptions of how much effort was actually exerted on tasks (as distinct from measures of motivation or willingness to exert effort, which were summarized in “Evaluation”; Baranski et al. 2004; Volkow et al. 2006). Behaviorally, execution is generally operationalized as speed of response on a task offering a reward (Beierholm et al., 2013; Croxson et al., 2009; Guitart-Masip et al., 2014, 2012a, 2011). For example, we consider reaction times (RTs) on the MID task a measure of execution. Studies that investigate neural correlates of execution use fMRI to measure BOLD response during execution of tasks required to obtain reward (Croxson et al., 2009; Dean et al., 2016; Schmidt et al., 2012).

D. Execution - Summary of studies

i. Execution - “simple invigoration” – Self-report measures

Although dopaminergic stimulants generally increase self-reported overall feelings of “vigor”, we examined only self-reports of vigor directed at attaining a reward. Two studies have tested such measures. In one, consistent with our hypothesis, modafinil (20mg) increased retrospective self-reported ratings of the effort individuals put into a rewarded task (Funayama et al., 2014). However, in a second, bupropion (150mg) had no effect on a similar retrospective rating of effort (Ikeda et al., 2019).

ii. Execution - “simple invigoration” – Behavioral measures

The nine studies that have utilized behavioral measures generally do not support the hypothesis that increasing DA should increase speed or vigor of responses made for reward. Only one study was wholly consistent. In this study, participants needed to stay above a certain grip strength threshold to earn a reward (Michely et al., 2020). The authors measured “excess” grip strength applied beyond the threshold, on the basis that this represents implicit, unconscious invigoration, rather than explicit decisions about how to allocate effort. L-DOPA (150mg) increased “excess” grip strength in high reward trials, as compared to low or uncertain reward trials. In contrast, seven studies reported no effect of increasing DA on reaction time. In seven studies reporting raw reaction times on the MID or a similar task, modafinil (20 mg), L-DOPA (150mg and 100mg), methamphetamine (20mg), methylphenidate (0.5 mg/kg) and bupropion (150mg) did not affect the speed of responses on reward trials vs. non-reward or loss trials (Apitz and Bunzeck, 2014; Funayama et al., 2014; Graf et al., 2016b; Ivanov et al., 2014; Mayo et al., 2019; Weis et al., 2012; Wittmann et al., 2015). In a sub-acute study by Dean et al. (2016), augmenting DA over the course of a week (bupropion, 150 mg daily) also did not alter response speed in a button-pressing task to either obtain a pleasant taste or avoid an aversive taste. A single study suggests that some of these null results could be due to failure to account for baseline levels of DA. This study found that augmenting DA using bromocriptine (1.5mg) sped performance on rewarded trials of a reaction time task for individuals with the A1A1 DRD2 genotype, which is associated with reduced DA transmission at baseline, while impairing performance in all other genotypes (Kirsch et al., 2006). In summary, the effects of DA augmentation are generally null, although one study suggests a more complex inverted-U pattern in which only individuals with low baseline DA may be invigorated by DA increasing drugs.

The seven studies examining the effect of reducing DA similarly were dominated by null results. Only one was consistent with our broad hypothesis that reducing DA should reduce speed of rewarded responses. In this study, reducing DA using an atypical antipsychotic (olanzapine, 5mg) slowed reaction times to rewarded trials specifically (Abler et al. 2007). In contrast, three studies reducing DA using amino acid mixtures (AMPT, APTD) found no effects on speed of rewarded responses (Bjork et al., 2014; da Silva Alves et al., 2011; Nagano-Saito et al., 2012). Risperidone (0.5mg) also did not change the speed with which participants reached a pre-specified grip strength required to obtain reward (Fiore et al., 2018). Haloperidol (1.5mg) did not significantly reduce “excess” grip strength in the same grip strength paradigm described above (Michely et al., 2020). Haloperidol (2mg) also did not slow responses to rewarded vs. non-rewarded trials on a go/no-go task (Schutte et al., 2020b). This evidence suggests that augmenting and reducing DA in healthy adults has limited effects on simple vigor of responses for reward.

iii. Execution - “simple invigoration” – Neuroimaging measures

Two pharmaco-fMRI studies have investigated brain activity during execution of reward-related behaviors. As described above, Dean et al. (2016) used sub-acute administration of bupropion (150mg daily) over the course of a week, and required participants to complete either an easy or hard task to either gain a pleasant taste or avoid an unpleasant one. Bupropion increased BOLD signal in the caudate, ventromedial prefrontal cortex, dorsal ACC/paracingulate gyrus and putamen during easy trials to obtain a pleasant taste, compared to hard trials to obtain a pleasant taste. The selected contrast, between high and low effort reward trials, makes these results somewhat difficult to interpret, but broadly, increasing DA levels did potentiate activity in DAergic and reward-related areas when required effort was low (Dean et al., 2016).

Regarding DA reduction, Fiore et al. (2018) examined activity in the internal and external globus pallidus, and substantia nigra pars reticulata during the exertion of grip for reward, under risperidone (0.5mg). They found that risperidone decreased activity during low reward (vs. high reward) trials in the external globus pallidus, but increased activity to low reward (vs. high reward) trials in the substantia nigra. This complex regional interaction effect suggests that finer parsing of the striatum may be necessary to understand the effects of DA on execution of actions to gain reward.

iv. Execution - “average reward rate” – Behavioral measures

As noted above, Niv et al. suggested that DA levels represent the average local reward rate as an indicator of opportunity cost, rather than being related to the reward available for a given trial (Niv et al., 2007). This would predict a differential effect of DA manipulations based on average rate of reward available/opportunity cost, rather than the reward available for a given trial. Only two studies have tested this hypothesis, both using DA augmentation. The first was consistent with this hypothesis. In Beierholm et al. (2013), participants were presented with a varying reward and required to respond to a simple cognitive effort task within 500ms to attain the reward. Computational modeling of responses found that average reward rate was positively related to response times, and this relationship was stronger after augmenting DA with L-DOPA (150 mg). However, these findings contrast with those reported by Zenon et al. (2016). In this study, participants were required to complete two types of handgrip tasks in exchange for monetary reward. In one, exerting more effort allowed faster completion, and thus effort was related to opportunity cost. In the other, the time it took to complete the task was fixed, but amount of reward received was proportional to the amount of effort exerted, so effort was unrelated to opportunity cost. Contrary to what would be expected based on the theory of Niv et al., augmenting DA (L-DOPA, 125mg) increased effort on the task where more effort directly related to more reward, but did not affect effort when exerting more effort allowed the person to save time (Zenon et al., 2016). The authors interpreted these findings as indicating that DA may change evaluation of the cost of effort vs. reward (and these results were thus also discussed in “Evaluation”), but does not alter vigor. These studies differed in a number of ways (task complexity, cognitive vs. physical effort requirements) making them difficult to compare, but based on this small literature, there are conflicting findings regarding whether DA manipulation in humans affects vigor by enhancing a DAergic signal representing average rate of reward.

v. Execution - “go to win” – Behavioral measures

As described above, the possibility that DA controls a Pavlovian, prepared “go to win” response can be investigated in orthogonalized tasks that contain both “go” and “no-go” trials to both obtain reward and avoid loss. Two studies have investigated such tasks with null results. In one, augmenting DA with L-DOPA (150 mg) sped response times to all go trials on an orthogonalized go/no-go task, regardless of outcome valence, suggesting a simple psychomotor effect rather than “go to win” enhancement (Guitart-Masip et al., 2012a). In another, Swart et al. found that methylphenidate (20mg) did not speed reaction times overall, or on to “go to win” trials specifically. Of note, the task used by Swart et al. also required learning of action and outcome contingencies, which may have affected results (learning outcomes from this task will be discussed in “Learning”).

Finally, one study examined DA reduction. This study used a Pavlovian-to-instrumental transfer design to investigate the effect of DA on conditioned associations between reward cues and action (Weber et al., 2016). Following classic animal paradigms, healthy adults were first trained to press a button for a chocolate candy reward. They then completed Pavlovian conditioning in which one visual stimulus (the CS+) was associated with delivery of the chocolate reward, and one visual stimulus (the CS−) was presented the same number of times without reward delivery. Finally, the two visual stimuli were re-presented during a period in which participants were free to press the button, but no rewards were delivered. During this period, there was less button pressing to the CS+ under amisulpride (400mg), consistent with the idea reducing DA impairs Pavlovian cue-action associations. In summary, there are comparatively few studies evaluating the “go to win” theory, and the disparate results and outcome measures used (reaction times, accuracy, Pavlovian-to-instrumental transfer) make it difficult to draw conclusions.

E. Execution - Summary, conclusions, recommendations

Overall, there is less evidence for the role of DA in the execution and invigoration of goal-directed behaviors. Little self-report work exists. There is also a lack of studies using imaging to elucidate the neurobiology of response vigor in response to DAergic manipulations. Those that do, suggest that finer anatomical resolution may be needed to capture the complex effects of DA on execution for reward. Thus, most of the evidence comes from behavioral studies. Regarding the “simple invigoration” theory, the majority of behavioral findings are null, suggesting research should focus on the more sophisticated “average reward rate” and “go to win” hypothesis. However, the available behavioral evidence testing whether vigor is linked to a DAergic signal capturing average reward rate yields conflicting results, as do the few behavioral studies investigating the “go to win” hypothesis.

There are notable weaknesses in existing behavioral evidence. First, the majority of behavioral studies reviewed under the “simple invigoration” theory were providing secondary results in the context of fMRI, so these tasks might not be particularly sensitive to behavioral changes. Second, we compared behavioral results from a variety of measures with different types of response requirements and structures. The “go-to-win” theory suggests that different types of response requirements may not be comparable – in particular, performing an action may differ qualitatively from having to withhold an action to gain a desired outcome.

V. Pleasure

A. Pleasure - Definition

We label the fourth stage of reward processing, as outlined in Figure 1, “Pleasure”. We define pleasure as the hedonic impact of receiving a reward, which has also been called “liking” or “consummatory reward” (Berridge, 1996). The hedonic impact of anticipating a future reward was covered in “Anticipation”, and adjustments made to behavior based on receiving a rewarding outcome are covered in “Learning”.

B. Pleasure - Predictions based on putative neurobiology

Unlike the other processes covered in this review, the known neurobiology of pleasure does not suggest a strong role for DA. Although DA was originally thought to play an important role in pleasure (Olds and Milner, 1954), current evidence shows that opioid “hot spots” in the ventral striatum, pons, and orbitofrontal cortex, are responsible for the hedonic effects of rewards (Barbano and Cador, 2006; Berridge and Kringelbach, 2015; Colasanti et al., 2012). Although these hot spots are co-located with DA reward-related circuitry, and may have mutual influence with DA circuits, they are opioidergic (Castro and Berridge, 2014; Ho and Berridge, 2013).

Consistent with this, in animals, opioid agonists such as morphine increase the hedonic liking reaction to sweet and even bitter tastes (Barbano and Cador, 2006; Doyle et al., 1993; Murphy et al., 1990), and this occurs in the hedonic hot spots in the NAcc and ventral pallidum (Smith and Berridge, 2007). However, dopaminergic drugs do not affect taste palatability in rodents – for example, pimozide had no effect on liking reactions in rats (Peciña et al., 1997) and tetrabenzadine similarly did not affect hedonic responses (Pardo et al., 2015). These findings suggest that dopaminergic drugs may have less effect on liking or pleasure compared to other aspects of reward (Berridge and Kringelbach, 2015).

Based on the putative neurobiology of pleasure, we expect that DA augmentation and reduction should have comparatively less effect on pleasure, in contrast to other reward-related functions. However, we note that in typical fMRI paradigms it is difficult to separate the hedonic impact of receiving a reward from other processes with DAergic elements, including reward learning. Therefore, we will note when imaging studies included measures of self-reported pleasure, and particularly when these were correlated with imaging results.

C. Pleasure - Measures

Pleasure in humans has been assessed using self-report, behavioral and neuroimaging measures. Self-report measures include the Profile of Mood States (POMS) subscales of elation, positive mood, or total mood disturbance (McNair et al., 1971), and Addiction Research Center Inventory (ARCI), which measures prototypical effects of abused drugs (Martin et al., 1971). In this review, the ARCI generally appears in studies where the goal is to measure how one drug moderates the pleasurable effects of another drug, and the two subscales of most interest are the amphetamine (A) scale and morphine-benzedrine (MBG) scale, which both capture euphoric drug effects. Pleasure can also be measured by VAS ratings after receiving a reward, e.g. of happiness, liking and enjoyment. Behaviorally, electromyography (EMG) of the facial muscles (i.e., zygomatic “smile” muscle and corrugator “frown” muscle”) can measure small facial movements in response to receiving rewards. Neuroimaging of pleasure has generally used fMRI to measure BOLD response at the time of receiving a reward. ERPs are also commonly used to measure brain activity during reward receipt; however, most ERP studies employ reward-learning paradigms comparing receipt to anticipation, rather than focusing on primarily on activity at the time of receipt of reward, and thus are covered in the “Learning” section.

While many studies of pleasure use monetary outcomes, other rewards have also been used, e.g., sweet tastes, food pictures, erotic images, drugs of abuse, so type of reward will be noted as a potential moderator.

D. Pleasure - Summary of studies

i. Pleasure – Self-report measures

While it is well documented that many DA-releasing drugs induce euphoria (Boileau et al., 2007; Brauer and de Wit, 1995; Childs and de Wit, 2009; Clatworthy et al., 2009; de Wit et al., 2002; Leyton et al., 2002; Sevak et al., 2009; Weafer et al., 2017; White et al., 2006), here we focus on how dopaminergic drugs affect pleasure in response to other rewards. There have been five studies examining effects of DA augmenting drugs on self-reported pleasure in response to rewards. Only one showed the null effects we would predict. In this study, liking of pleasant and unpleasant tastes was unchanged by 150mg bupropion (Dean et al., 2016). In contrast, two other studies showed a “flattening effect” -- under placebo, pleasure (i.e., excitement or happiness) was higher after receiving large monetary rewards compared to receiving small rewards or losing (Knutson et al., 2004; Rutledge et al., 2015), but under d-amphetamine (.25mg/kg) or L-DOPA (150mg), pleasure was similar after receiving small or large outcomes (and in fact, after wins vs. losses). In both cases, DA increasing drugs increased pleasure/excitement for events that were not previously pleasurable, while perhaps slightly decreasing responses to previously highly pleasurable events. Similarly, in another study d-amphetamine (20mg) increased pleasantness ratings equally across positive, neutral and negative emotional images. This is not exactly a “flattening” pattern, but again suggests (in a different reward type) that DA augmentation might boost the pleasurable effect of stimuli not typically considered pleasant (Wardle and de Wit, 2012). Finally, one study reported that L-DOPA (100mg) increased liking ratings in response to listening to music (Ferreri et al., 2019). Thus, the modal finding was suggestive of “flattening” of pleasure.

Results from 14 studies examining DA reduction were more supportive of our overall hypothesis that DA manipulations should not strongly impact pleasure. These results were also consistent across a variety of reward types. DA reduction with both APTD and haloperidol (3mg) did not significantly affect positive mood and elation in response to monetary wins in gambling tasks (Bjork et al., 2014; Zack and Poulos, 2007). Null results were also seen in two studies examining the effects of DA antagonists on the euphoric effects of other drugs. Pimozide (4mg) and fluphenazine (3mg and 6 mg) did not reduce the euphoric effects of d-amphetamine (Brauer and de Wit, 1995) and haloperidol (3mg) had no effect on the positive mood effects of methamphetamine (Wachtel et al., 2002). Similarly, three studies reported no effects of ATPD on subjective responses to d-amphetamine (Leyton et al., 2007), alcohol (Barrett et al., 2008), and cocaine (Leyton et al., 2005) when administered to individuals without substance use disorders. ATPD also did not alter responses to nicotine (Venugopalan et al., 2011). Reduction of DA (via APTD, 400mg sulpiride, and .25mg pramipexole) also had no effect on liking ratings of pleasant and unpleasant food (Frank et al., 2016; Hardman et al., 2012; McCabe et al., 2013, 2011); although, some of these null self-report effects were accompanied by significant changes in brain responses. One study reported that haloperidol (3mg) had no effect on affective ratings of sexual stimuli. Only one study reported an effect. Risperidone (2mg) decreased pleasure in response to listening to music (Ferreri et al., 2019). To summarize, DA reduction appears to have a small, if any, effect on self-reported pleasure in response to rewards, based on 13 out of 14 studies reporting null results.

ii. Pleasure – Behavioral measures

Only two studies have investigated the effects of DA drugs on pleasure using behavioral measures. In contrast to our hypotheses, both suggested that DA augmentation modulates physiological responses to reward. Specifically, d-amphetamine increased zygomatic (smile muscle) activity (at 10 mg) and decreased corrugator (frown muscle) activity (at 20 mg) to positive images, indicating enhancement of positive responses (Wardle and de Wit, 2012). More research using facial muscle activity is needed to confirm these results. Another study looked at the effect of L-DOPA (100mg) and risperidone (2mg) on electrodermal activity while listening to music. L-DOPA increased, while risperidone decreased, electrodermal activity during highly pleasing portions of music, compared to more neutral portions (Ferreri et al., 2019). These behavioral measures suggest DA manipulations may have a stronger effect on physiological or behavioral measures rather than self-report measures of pleasure, but more research is needed.

iii. Pleasure – Neuroimaging measures

DA augmenting drugs appear to have mixed effects in the eight neuroimaging studies that have investigated responses during receipt of rewards. Only one of these found null results consistent with our hypothesis. d-Amphetamine (.25mg/kg) had no significant effect on BOLD activity in any brain area in response to receiving monetary rewards (Knutson et al., 2004). Two studies reported results consistent with the “flattening” effect seen in self-reported pleasure. The first reported that bupropion (150mg) increased BOLD activity in the orbitofrontal cortex to both pleasant and unpleasant tastes, but reduced BOLD in the caudate to pleasant tastes (Dean et al., 2016). Ventral striatal and amygdala activity were also increased in response to unpleasant tastes only. No DA effects on self-reported liking were found in this study and the task contained multiple phases of reward; therefore, it is unclear whether this brain activity reflects pleasure or other task-related processes. In the second, DA augmentation (d-amphetamine; .5mg/kg) decreased dorsal ACC activity to sucrose, compared to water (Melrose et al., 2016). Yet while dorsal ACC activity was “flattened”, insular activity showed the opposite effect, increasing to sucrose compared to water under d-amphetamine.

Three studies using a variety of DA increasing drugs and rewards have reported results consistent with enhancement of brain activity during receipt of reward. Increasing DA with bupropion (150mg) increased activity in the NAcc, posterior midcingulate, mediodorsal thalamus, and amygdala in response to erotic videos (Abler et al., 2012, 2011), d-amphetamine (20mg) increased responses to receiving a monetary reward in the amygdala (O’Daly et al., 2014), and L-DOPA (100mg) increased responses to erotic images in the NAcc, and dorsal ACC (N. Y. L. Oei, Rombouts, Soeter, van Gerven, & Both, 2012). Of these studies, only Abler et al. (2011) reported relationships with subjective variables. Abler et al. (2011) found no correlation between subjective sexual functioning and the increase in response to erotic videos seen with bupropion.

Finally, one study reported decreased responses to reward. Methylphenidate (0.5 mg/kg) decreased activity in the mid occipital and left inferior parietal cortices to receipt of monetary reward (Ivanov et al., 2014).

Thus, for DA increasing drugs, imaging produces mixed results, with the brain areas and direction of effect inconsistent across studies. Further, because of task-related factors, such as the inclusion of an effort phase in Dean et al. (2016) or expectation and conflict manipulations in Ivanov et al., (2014), and the comparatively small number of studies investigating subjective responses in conjunction with imaging, it is difficult to discern if brain effects relate to pleasure or other processes. Thus, there is limited evidence that drug intended to increase DA enhance pleasure via changes in ventral striatal and/or orbitofrontal cortex activity.

Nine studies have examined the effect of DA reduction on brain responses to receipt of rewards. Three of these studies were consistent with our hypothesis in reporting no significant changes in BOLD response in any brain areas after DA reduction (Bjork et al., 2014; Graf et al., 2015; Sescousse et al., 2016), and one of these was accompanied by similarly null self-report findings (Bjork et al., 2014). Two of the null studies found that DA reduction using APTD and sulpiride (400mg) had no effect on BOLD activity in response to receiving monetary rewards (Bjork et al., 2014; Sescousse et al., 2016), while the third found no significant effect of amisulpride (200mg/day over 7 days) on BOLD to an erotic video (Graf et al., 2015). However, five of the remaining studies generally favor decreased activation, most commonly in the ventral striatum and prefrontal cortex, across a variety of interventions including APTD, “low dose” pramipexole (0.25 and 0.5mg), sulpride (400mg) and haloperidol (3mg), across food, monetary and sexual stimuli (Frank et al., 2016; McCabe et al., 2013, 2011; Oei et al., 2012; Riba et al., 2008). Notably, all three studies using food stimuli saw decreases, while other stimuli showed more mixed effects. However, the three food studies also saw no accompanying effects on self-report. Finally, one study reported that DA reduction with “low dose” pramipexole (0.5mg) actually increased activation of the NAcc in response to rewards (Ye et al., 2011). To summarize, modal results suggest DA reduction decreases activity in the striatum and frontal cortex during receipt of reward. Of note, the reward type most consistently accompanied by decreased BOLD response with DA reduction was food reward. Studies investigating other reward types had more inconsistent results. It is notable that reductions in brain activation in response to rewards do not always correspond to reductions in the hedonic experience of receiving a reward. Although more research is needed on the relationship between neural responses to reward and subjective experience, the findings to date suggested that these measures reflect different underlying processes, both involving DA. It is also possible that different outcomes are related to methodological factors because many processes are involved in both reward valuation (e.g., evaluation and learning) and in self-report measures (e.g., expectancies).

E. Pleasure - Summary, conclusions, recommendations

In conclusion, DA’s role in the hedonic impact of receiving a reward is still unclear. Self-report results indicated that increasing DA may “flatten” subjective pleasure, by increasing it in response to otherwise non-rewarding stimuli, and possibly slightly decreasing responses to highly rewarding stimuli. In contrast, decreasing DA appears to have little effect on self-reported pleasure. In general, both DA enhancement and reduction modify brain activity in response to rewards, most often in the NAcc, amygdala, dorsal ACC and orbitofrontal cortex, but the direction of effect is sometimes inconsistent. Both manipulation and reward type appear to have an impact here, as studies of DA reduction with food rewards had the most consistent findings of reduced activation. Further, it is often unclear if brain activity in response to receiving a reward actually reflects hedonic impact, as many studies do not report correlations between self-report ratings of pleasure and brain activity in response to receiving a reward, and those that do had largely inconsistent results when comparing self-report to imaging.

This summary suggests several recommendations for future research. First, researchers should include both subjective and objective measures when assessing pleasure in response to a reward. Studies that include both subjective and objective responses to rewards are sparse, and those that do, typically do not report the correlation between the two. One advantage of DA manipulation studies in humans is the ability to measure subjective impact of the underlying processes of interest. Second, more studies are needed that measure pleasure in response to a reward without confounding factors, such as manipulations of expectations. Simpler designs, especially when measuring brain activity, would help parse out brain responses to other processes such as learning and prediction. Third, varying levels of reward and loss should be included to assess whether effects on self-reported pleasure or brain responses change with the magnitude of reward/loss. Studies like this will allow for a more nuanced understanding of DA’s role in pleasure and may confirm or deny the “flattening” result reported above.

VI. Learning from Reward

A. Learning from reward - Definition

We label the fifth stage of reward processing, as outlined in Figure 1, “Learning from Reward”. We define learning as the process in which associations between actions and rewarding outcomes are established and expectations and behaviors are adjusted when outcomes differ from expectations (Schultz, 2016a). We also include here studies investigating “reward prediction errors” or RPEs, which refer to the difference between predictions about rewards and actual obtained rewards, as these are thought to be integral “teaching” signals for reward learning (Rescorla and Wagner, 1972; Sutton, 1988). Note that while we list learning last in Figure 1, learning occurs over time, typically over several trials, and can occur at any point in the reward process when new information becomes available (Sutton and Barto, 1998).

B. Learning from reward - Predictions based on putative neurobiology

Studies of the neurobiology of learning suggests that DA manipulations should affect learning in humans. Evidence suggests that phasic DA signaling in the VTA and NAcc represents a biological instantiation of the reward prediction error. Midbrain DA neurons initially fire in phasic bursts during receipt of rewarding stimuli such as food, sexual contact, and drugs of abuse (Bassareo et al., 2007; Hernandez and Hoebel, 1988; Kiyatkin and Rebec, 2001; Pfaus et al., 1990); however, when the reward is predicted by another stimulus, dopaminergic firing shifts backward in time to the predictive stimulus (i.e., cue), and instead of occurring when the reward is delivered. When a reward is unexpected or larger than predicted by a cue, DA signaling increases at the time of the received reward, indicating that the outcome was better than expected. When a reward is predicted, but then not delivered, the DA neurons suppress firing at the time of the expected reward, signaling the outcome is worse than expected. Thus, the phasic firing of midbrain DA neurons appears to code RPEs (Cohen et al., 2012; Pan et al., 2005; Zaghloul et al., 2009).

Consistent with this neurobiology, DA antagonists appear to blunt both Pavlovian and instrumental learning in animals (Wise, 2004). For example, animals that have already learned stimulus-reward associations will reduce their responses to the rewarding stimulus over several minutes or days after being treated with DA antagonists (McFarland and Ettenberg, 1995; Yokel et al., 1975). Animals who have not yet learned these associations and are treated with DA antagonists do not learn to press a lever for rewards and do not develop conditioned place preferences (Wise and Schwartz, 1981). Similarly, in a rat probabilistic reward task, response bias toward rewarded cues is increased after administration of d-amphetamine and decreased after “low dose” pramipexole (Der-Avakian et al., 2013). Taken together, these findings suggest that systemic administration of DA drugs should affect reward learning in humans.

Based on the putative neurobiology of reward learning, our hypothesis is that increasing DA should increase reward learning and promote RPE coding in the midbrain, striatum, and associated frontal cortex areas, while reducing or blocking DA should impair reward learning and disrupt neural coding of RPEs.

C. Learning from reward - Measures

Reward learning in humans has been assessed using behavioral measures and neuroimaging. Behavioral measures generally consist of reinforcement learning tasks, in which a pair of stimuli is presented and the participant must learn which stimulus is more likely to lead to reward, with percentage of correct choices (or accuracy) indicating learning (e.g., Pessiglione, Seymour, Flandin, Dolan, & Frith, 2006). Variants on this paradigm include reversal learning, in which reward rules change, and learning is quantified by the mean error rate, time to reach a specified learning criterion, probability of correct choice following a reversal, or probability of switching after receiving non-reward feedback. Another variant is learning Go/No-Go tasks, in which participants must learn which stimulus requires a response or withholding of a response to receive or avoid losing a reward (Guitart-Masip et al., 2012b), with variables of interest including the probability of correct choice for go trials and probability of correct omission for no-go trials. Other popular tasks include the probabilistic reward task (PRT; Pizzagalli, Jahn, & O’Shea, 2005; Tripp & Alsop, 1999), which assesses development of response bias for a more highly rewarded stimulus utilizing signal detection theory, and time estimation tasks, which require adjusting responding according to feedback (e.g., Zirnheld et al., 2004). Measures from the above designs will be referred to as “raw” learning measures, as they are direct measures of task metrics and do not reflect a theoretically driven computational model.

Computational modeling, which models the fit of observed data to reinforcement learning algorithms, is often used to analyze effects of DA manipulations on these tasks (Rescorla and Wagner, 1972; Sutton and Barto, 1990). Here we briefly review the most frequently examined learning parameters in these models. The main assumption of a standard reinforcement-learning model is that the goal of learning is to predict the sum of future rewards based on the perceptual stimuli available. Reward prediction errors are available immediately after each reward trial and help estimate the actual sum of future rewards. In the standard model, the prediction error δ(t) is the difference between actual and expected outcomes, and serves as a teaching signal for the next trial: δ(t) = R_t + V(S_t) − V(S_t−1), where R_t is the actual value of the present outcome (at time t) and V(S_t) and V(S_t−1) are predictions about upcoming rewards at time t and time t-1 (the prediction in the previous state)¹. The estimate V(t) of all future rewards is updated using the prediction error δ(t) and a constant learning rate α to scale the size of the update: V(t)new = V(t)old + α*δ(t). In another model referred to as a basic Q learning algorithm, prediction errors are also used to update the expected value of choosing a particular action (e.g., Samejima, Ueda, Doya, & Kimura, 2005). In this case, the model estimates the expected value of choosing from two actions or stimuli Q_a or Q_b (which typically begin at 0 on trial 1) and updates the expected value by the prediction error: Q_a(_t+1) = Qa(_t)+ α*δ(t). The probability P of choosing a or b can then be modeled by using the estimated expected values for Q_a and Q_b. Typically, the softmax rule is employed such that:

$$P_{\text{a}}\left({}_t\right)=\frac{\text{exp}\left(Q_{\text{a}}\left({}_t\right)/\beta \right)}{\left(\text{exp}\left(Q_{\text{a}}\left({}_t\right)/\beta \right)+\text{exp}\left(Q_{\text{b}}\left({}_t\right)/\beta \right)\right).}$$

β is another constant, referred to as the temperature, which represents the participant’s biases towards random choice or choice of the highest value. Both learning rate and temperature are free model parameters that can either be set or compared between conditions. These models are modified to test specific predictions about how reinforcement learning networks function. Then, model fit is assessed to test if the model explains the observed data well. For the purpose of this review, we will focus primarily on whether DA manipulations affect α and β, as these are the most commonly examined parameters with the best accumulation of evidence.

Reward prediction errors have also been operationalized as brain activity in response to rewards on tasks that involve learning or making predictions about upcoming rewards. There are two primary ways RPEs are operationalized in imaging: 1) though BOLD activity that is correlated with the prediction error δ(t) estimated from the computational model and 2) through the error-related (ERN) or feedback-related (FRN) negativity ERP components measured using EEG. The ERN and FRN occur 0–150 ms and 250–250 ms post behavioral error or error feedback/unexpected outcomes, respectively (Gehring and Fencsik, 2001; Miltner et al., 1997; Scheffers et al., 1996). It is hypothesized that the ACC generates the ERN/FRN (Dehaene et al., 1994). Finally, one MEG study measured both event-related fields (ERFs) and oscillatory responses to reward feedback following differing reward expectations.

D. Learning from reward - Summary of studies

i. Learning from reward – “Raw” behavioral measures

We hypothesized that increasing DA should generally improve reward learning, but results from 18 studies are highly mixed, with only two studies supporting the most straightforward version of this hypothesis. In one, L-DOPA (100mg) improved learning of an association between a stimulus and monetary reward (but only when compared to a DA antagonist, not compared to placebo; Pessiglione et al. 2006). Similarly, in another study where participants had to make judgments about somatosensory stimuli and were rewarded when those judgments were correct, L-DOPA (100 mg) increased accuracy overall compared to placebo and haloperidol (Pleger et al., 2009). Thus, only two studies found that increasing DA with L-DOPA increased reward learning.

Seven studies indicated that administration of a DA-enhancing drug did not significantly affect “raw” learning rates on reversal learning or probabilistic learning tasks (Bellebaum et al., 2017; Bernacer et al., 2013; Diederen et al., 2017; Dodds et al., 2008; Jocham et al., 2011; Weis et al., 2012). These studies employed a variety of medications (i.e., modafinil, methylphenidate, methamphetamine, and “low dose” amisulpride) and a variety of tasks, including reinforcement learning, reversal learning, and probabilistic selection tasks.

Finally, two studies found results directly contradictory to our hypothesis. One reported that the agonist bromocriptine (1.25mg) produced worse reversal learning, although this was only significant compared to the antagonist sulpride, not placebo (Van Der Schaaf et al., 2012). The second reported worse learning on a reversal learning task under L-DOPA (100mg) compared to placebo, which the authors interpreted as consistent with an inverted-U hypothesis, suggesting “too much” DA can be detrimental, although baseline differences were not explicitly tested in this study (Vo et al., 2016).

Indeed, our straightforward hypothesis -- that increasing DA should increase reward learning -- does not take into account possible baseline differences or inverted-U effects, which have been commonly examined in this area. Five studies have considered the role of baseline DA in DAergic modulation of learning, with three supportive of an inverted-U effect and two supportive of a “rich get richer” effect. Supportive of the inverted-U hypothesis, both methylphenidate (60mg) and bromocriptine (1.25mg) improved reward learning in those with less baseline DA in the caudate and ventral striatum (as measured by PET), and impaired learning in those with more baseline DA (Clatworthy et al., 2009; Cools et al., 2009). Similarly, “low dose” sulpiride (200mg) impaired learning from feedback in extraverts compared to introverts (Mueller et al., 2014b), also broadly supportive of an inverted-U model although employing a less precise proxy measure. Yet others have reported potential “rich get richer” effects. In one study, methylphenidate increased reward learning in those with high baseline working memory and decreased it in those with low working memory (Van Der Schaaf et al., 2013). Another study found that learning rate was increased under cabergoline (1.5 mg), but only for those with the A1- DRD2 Taq1a genotype associated with increased DA (Cohen et al., 2007).

Finally, three studies have investigated the more nuanced hypothesis that DA may specifically facilitate learning associations between taking action and gaining reward, or “go-to-win” learning (Guitart-Masip et al., 2014). Only one was supportive -- “low dose” haloperidol (2mg) increased go and decreased no-go learning. However, this effect was seen only in those with low working memory (thought to be indicative of lower DA tone; Frank and O’Reilly 2006), suggesting an inverted-U effect. Of note, this same dose of haloperidol (2mg), employed as an intended reduction strategy, has also reduced reward learning (these findings will be discussed in the upcoming section). One study found null results of methylphenidate (20mg) on “go to win” learning (Swart et al., 2017). However, although learning was unaffected, Swart et al. (2017) did find that methylphenidate increased a general “go-to-win” bias in responses, particularly in those with high working memory span (Swart et al., 2017). These authors suggested that this might represent DA increasing the extent to which learning relies on prepared, Pavlovian (i.e., “go-to-win”) systems while not strongly affecting instrumental learning. Finally, one study found contrasting results. Contrary to the hypothesis that DA should enhance “go-to-win” learning, L-DOPA (150mg) instead reduced the bias towards “go-to-win” responding in the learning version of the go/no-go task (Guitart-Masip et al., 2014). These authors suggested that this might represent DA reducing the extent to which learning relies on prepared, Pavlovian (i.e., “go-to-win”) systems and enhancing the influence of instrumental, executive control systems, in contrast with the interpretation of Swart and colleagues. To summarize, “raw” results on reward learning tasks with drugs intended to increase DA levels have produced highly mixed results and divergent interpretations.

In contrast to the mixed effects for drugs intended to increase DA, DA reduction seems to more consistently impair reward learning. A total of 19 studies have investigated the effect of DA reduction on reward learning, with eleven supporting the hypothesis that DA reduction decreases learning. For example, administration of haloperidol (1mg) led to less money won and less rewarding choices on an instrumental learning task, although this effect was not significantly different from placebo, only from L-DOPA (Pessiglione et al., 2006). Similarly, during a rewarded somatosensory judgement task, haloperidol (2mg) decreased accuracy overall and eliminated magnitude of reward effects on accuracy (Pleger et al., 2009). Reducing DA (3 mg haloperidol) also decreases accuracy on time estimation tasks (Forster et al., 2017; Zirnheld et al., 2004), reversal learning tasks (400–800 mg sulpiride; Janssen et al., 2015) reward prediction tasks (600mg sulpiride; Diederen et al., 2017), and rewarded go/no-go tasks (APTD; Leyton et al., 2007). Consistent results have been reported for amisulpride (400mg), memantine (20mg; Jocham, Klein, & Ullsperger, 2014) and “low dose” pramipexole (.5mg; Pizzagalli et al. 2008b; Santesso et al. 2009b). One study measured slot-machine gambling behavior and found that under placebo, healthy adults showed a consistent positive relationship between payoff and bet size on the subsequent trial. Under haloperidol (3mg), this relationship was unstable, with the correlation between pay off and bet size reducing over time (Tremblay et al., 2011). While slot machines differ from typical learning designs, as there is no real relationship between winning and betting, these results suggest that DA reduction affected instrumental learning aspects of gambling behavior.

However, seven studies also reported null effects. One theme is that while DA antagonists appear to impair reward learning fairly consistently, DA depletion through APTD does not seem as effective. Apart from one study (Leyton et al., 2007), the majority of APTD studies report no effect on reversal learning, instrumental learning, investment learning tasks, or gambling tasks (de Wit et al., 2012; Robinson et al., 2010; Sevy et al., 2006; Tobia et al., 2014). In addition to the null findings with APTD, two studies reported no effects of antagonists (sulpride 400mg and 800mg) and one no effect of “low dose” (0.25 and 0.5mg) pramipexole on reward learning for monetary outcomes (Brom et al., 2016; Dodds et al., 2008; Eisenegger et al., 2014; Hamidovic et al., 2008). Finally, one study measured the effect of haloperidol (3mg) on conditioned sexual arousal and found no effects on reward learning (Brom et al., 2016). The significant results reported for antagonists above were all with monetary reward, so this null finding also suggests these effects may not generalize to non-monetary rewards.

The above studies did not measure baseline individual differences in DA levels or putative proxies such as working memory capacity. The two studies that did, reported that DA reduction (“low dose” cabergoline, 1.25mg; sulpride 400mg) actually increased reward learning in those with greater working memory capacity (Frank and O’Reilly, 2006; Van Der Schaaf et al., 2012), consistent with an inverted-U hypothesis.

In summary, antagonists appear to consistently decrease reward learning, while agonists and dietary manipulations have mixed to null results. A single study suggests that this effect of antagonists may be limited to monetary rewards, but more investigation of reward type is needed. Two studies that investigated baseline differences also provided support for a non-linear, inverted-U relationship.

ii. Learning from reward - Computational modeling measures

As computational models are flexible, we attempt to synthesize studies that have employed similar algorithms, such as standard reinforcement learning models. Here, we summarize studies that measured the effect of a DA enhancing or reducing drug on the learning rate (α) parameter and the temperature (β) parameter. It should be noted that other studies exist that employ reinforcement learning models, but hold α and β constant, and as such, those will not be reviewed here.

Six studies have measured the effect of a DA enhancing drug on α and none of them support the hypothesis that increasing DA should increase α. Four studies that looked at α reported no effects with DA enhancement (Diederen et al., 2017; Guitart-Masip et al., 2014; Jocham et al., 2011; Wunderlich et al., 2012). Notably, Diederen et al (2017), Guitart-Masip et al. (2014), and Wunderlich et al (2012) utilized modified versions of reinforcement learning algorithms to specifically test other model parameters, which could explain the null results on α. One study reported that increasing DA with bromocriptine (1.25mg) increased α in those with low striatal DA (measured with PET) and low working memory; this drug effect decreased as DA synthesis capacity increased, suggesting an inverted-U shape relationship (Cools et al., 2009). One study found contradictory results. Bernacer et al. (2013) reported that methamphetamine (.3mg/kg) decreased α, and further α was restored to normal when methamphetamine was taken with the DA antagonist amisulpride (400mg). Thus, taken together, it appears unlikely that increasing DA improves α. In terms of the temperature parameter β, all six studies reported null results (Bernacer et al., 2013; Diederen et al., 2017; Guitart-Masip et al., 2014; Jocham et al., 2011; Wunderlich et al., 2012). Therefore, we conclude that dopamine enhancing drugs also seem to have little to no effect on noise in responding.

DA reduction also has not shown strong effects on α, as all four studies reported null effects of DA reduction on model-derived learning rate (Diederen et al., 2017; Eisenegger et al., 2014; Jocham et al., 2014; Tobia et al., 2014). While DA reduction may not have a strong effect on α, it may alter β instead. Eisenegger et al. (2014) reported that the temperature (β) was higher under sulpiride (800mg), suggesting that decreasing DA simply increases the noise in the participant’s responding such that choices are less correlated with subjective value. However, this was the only study that reported an effect with β, the remaining three studies reported null results (Diederen et al., 2017; Jocham et al., 2014; Tobia et al., 2014). To summarize, the modal results is that DA enhancing and reducing drugs have little to no effect on α and β when using reinforcement learning models.

iii. Learning from reward – Neuroimaging measures

As it is thought that RPEs are DA dependent, augmenting DA functioning would also be expected to increase the brain activity thought to represent RPEs. Across the studies reported here, RPE activity is most often observed in the striatum (most commonly ventral), frontal cortex (ventromedial prefrontal cortex/orbitofrontal cortex and ACC), amygdala, and midbrain (Bernacer et al., 2013; Diederen et al., 2017; Jocham et al., 2011; Pleger et al., 2009). A total of ten studies have investigated the effects of drugs intended to increase DA on these brain areas during reward learning. Four of these are supportive of our hypothesis. In one study, increasing DA via L-DOPA (100mg) increased RPE activity within the ventral striatum, which was accompanied by increased reward learning (Pessiglione et al., 2006). Similarly, L-DOPA (100mg) increased activity in the OFC and VS in response to monetary reward signaling a correct responses on a somatosensory judgement task (Pleger et al., 2009). “Low dose” amisulpride (200mg), thought to boost D1 signaling, also enhanced RPE coding in the striatum and frontal cortex, with ventromedial prefrontal cortex activity specifically correlating with overall learning (Jocham et al., 2011). Finally, another study found that methylphenidate (60mg) decreased striatal BOLD during switch after negative feedback trials and increased frontal BOLD during stay after positive feedback, suggesting changes in DA facilitates behavior change (Dodds et al., 2008).

Four studies also reported null results. Two with bromocriptine (1.25mg/2.5mg), one with L-DOPA (100mg), and one with bupropion (150mg) found that DA manipulation had no effect on brain activity to rewarding feedback (Diederen et al., 2017; Graf et al., 2016a; Van Der Schaaf et al., 2012; Weis et al., 2013). Of note, in the study by Van der Schaaf et al (2012), the bromocriptine condition was associated with less activity in the VS in comparison to a sulpiride condition; however, bromocriptine did not differ from placebo.

Conversely, two studies reported that DA augmentation decreased RPE activity. One study reported that methamphetamine (0.3mg/kg) reduced model-derived RPE activity in the ventral striatum; however, in this study administration of methamphetamine was accompanied by mild psychotic symptoms, potentially contributing to the differing patterns of brain activity (Bernacer et al., 2013). In another study, methylphenidate (40mg) also decreased RPE BOLD activity in the ventral striatum. In this case the authors speculated that methylphenidate might have increased tonic DA, leading to a decrease in phasic DA (Evers et al., 2017). To summarize, four studies support the hypothesis that DA augmentation increases RPE brain activity, but results were mixed with four showing null effects and two showing contradictory effects.

If DA is necessary for RPEs, then as a corollary, reduction of DA should lead to reductions in brain activity thought to represent RPEs. Results from almost every study (six studies) in this section are consistent with this hypothesis. DA reduction (sulpiride 600mg; amisulpride 400mg; haloperidol 2mg; APTD) disrupts RPE activity most commonly in the ventral striatum and caudate (Diederen et al., 2017; Jocham et al., 2014; Pessiglione et al., 2006; Tobia et al., 2014). DA reduction (sulpiride 600mg; APTD) may also disrupt RPE activity in the amygdala and midbrain (Diederen et al., 2017; Tobia et al., 2014). However, one combination drug study reported that sulpiride (400mg) did not alter the complex pattern of methylphenidate (60mg) effects seen in the same study (Dodds et al., 2008).

A sole study indicated that effects of DA reduction on fMRI outcomes may also depend on working memory (again used as a proxy for baseline levels of DA). In this single study, those with higher working memory showed an increase in striatal activation in response to rewards after administration of a DA antagonist (along with an increase in learning, as described in the behavioral section). This is consistent with the inverted-U hypothesis, suggesting individuals with high working memory were potentially moving from a hyperactive DA state to a more optimal state (Van Der Schaaf et al., 2012). However, this is the only study in this section that reported on baseline working memory; therefore, the modal result is that DA reduction simply disrupts RPEs.

We also expected that DA should modulate the ERN and FRN, which are thought to represent RPEs in the ACC (Holroyd and Coles, 2002; Proudfit, 2015). One study using d-amphetamine (15mg) suggested that DA agonists straightforwardly increase the ERN/FRN (de Bruijn et al., 2004). However, three studies using “low dose” sulpiride (200mg) suggest an inverted-U relationship. Of note, two of these examined the same participants using different proxies of DA functioning. All found “low dose” sulpiride enhanced the FRN in individuals with presumably low baseline DA levels (COMT Val/Val genotype, introverts – although see note above about possible conflicting effects of COMT genotype in different brain areas), but decreased it in individuals with high baseline DA levels (COMT Met allele carriers, extraverts) (Mueller et al., 2014b, 2014a, 2011). Finally, one study reported null effects of L-DOPA on ERFs (Apitz and Bunzeck, 2014).

Five studies have investigated the effect of DA reduction on the ERN/FRN, with three in support of our hypothesis. Antagonists decrease the amplitude of the ERN/FRN (de Bruijn et al., 2006), with accompanying effects on learning (Santesso et al., 2009; Zirnheld et al., 2004). Still, two papers have reported null effects of dopaminergic drugs on the FRN (APTD; Larson et al. 2015; 3mg haloperidol; Forster et al. 2017), one employing APTD and the other administering haloperidol. Notably, the APTD study did not use a learning task per se, so this could explain the null result.

E. Learning from reward - Summary, conclusions, recommendations

Across “raw” learning metrics, computational modeling outcomes and imaging measures of the RPE, drugs intended to increase DA and decreasing DA via precursor depletion appears to have mixed effects, while DA reduction using antagonists more consistently affect reward learning. Regarding the potential inverted-U relationship between DA and learning, evidence for this is difficult to evaluate due to characteristics of the literature. Specifically, few studies utilized the same drug manipulations, making it difficult to compare doses across studies, and no studies incorporated the multiple doses required to establish a dose-response curve. Further, studies differed significantly in how they established baseline DA functioning. While a minority used PET, more used working memory or other behavioral or self-report indicators. Within studies that used similar “proxy” measures such as working memory, some detected inverted-U patterns while others did not, but it is difficult to know whether to attribute this to the inexactness of the proxy measures. Relatively more consistently, the neuroimaging literature supports the role of DA in the generation of reward prediction errors, with augmentation increasing coding of RPEs and reduction decreasing RPEs in general.

However, complicating the picture, while computational modeling shows support for DA’s role in producing reward prediction errors in humans, there is limited evidence that DA modulation consistently affects model-derived learning rate. In response to this, authors have suggested (and calculated) additional model parameters that may be affected by DA, such as the balance between Pavlovian and instrumental learning (Frank and O’Reilly, 2006; Guitart-Masip et al., 2014; Santesso et al., 2009; Swart et al., 2017), PE scaling (Diederen et al., 2017), among others (Cools et al., 2009; Jocham et al., 2014; Tobia et al., 2014; Wunderlich et al., 2012). Because of the comparatively small number of papers examining each of these additional parameters, we have not reviewed these results here. However, it is possible that the unclear effect of DA augmentation on learning rate could potentially be due to failure to capture these more complex parameters. Computational modeling shows strong promise in highlighting the more complex ways DA may interact with learning, but we also note that comparability has been strongly hampered by a multiplicity of models. Calculating and reporting drug effects on both the simpler (and thus more likely comparable across studies) models along with more complex ones is recommended.

VI. Conclusions

A. Final Summary

The primary goal of this review was to answer the question: What can drug challenge studies in healthy human adults tell us about the role of DA in human reward processing? We draw three main conclusions from the review: i) DAergic drugs have markedly different effects on different phases of reward processing; ii) the relationship between DA and any given reward function appears unlikely to be linear; iii) the effects of DAergic drugs on reward-related behaviors vary depending on the type of outcome measure used (i.e., self-report, behavioral, and neural).

First, we found that the effects of DAergic drugs vary across different phases of reward processing. DAergic manipulations appear clearly to impact anticipation of rewards, and particularly neural measures of anticipation. DA also appears involved in the evaluation of risk-related costs vs. benefits when there is the possibility of loss, and the evaluation of effort costs vs. benefits. However, DAergic drugs had little consistent impact on reward-related decision-making when the cost involved was time. We found, as expected, that decreasing DA had little effect on self-reported pleasure, although increasing DA in some cases appeared to lessen the distinction in responses between pleasant and unpleasant stimuli. We found little evidence that DAergic drugs affected the speed or vigor of actions taken to attain reward, although many studies considered may not have been powered to measure behavioral vigor, as most were incidental measures taken during imaging studies. Finally, the role of DA reward learning appears complex, and may depend on the manner in which learning is assessed. DA antagonists in particular decreased behavioral measures of learning and neural correlates of prediction errors (both with fMRI and EEG) in the striatum and sometimes in the anterior cingulate, but DA agonists did not have a consistent effect.

These findings indicating both the distinctness of different reward functions, and the pattern of greater and lesser DAergic influence on these functions may help us to understand deficits in psychiatric conditions. For example, our findings are consistent with theories suggesting that “anhedonia”, as experienced in depression and schizophrenia, may have several distinct neurobehavioral components (Kring and Barch, 2014; Treadway and Zald, 2011b). Theorists have posited that anhedonia may involve reductions in anticipation of reward and willingness to exert effort for reward, as distinct from changes in ability to experience pleasure, and further suggested that anticipation and effort-related decision-making are more DAergically based (Treadway and Zald, 2011b). The idea that these functions are distinct and that anticipation and effort-related decision-making are more DAergically-based is quite consistent with our synthesis of findings here. These findings may also be relevant to understanding the effect of DAergic treatments, such as those commonly used for ADHD. Our synthesis would suggest that DAergic drugs may be more effective at ameliorating motivational aspects of ADHD rather than aspects of impulsivity that involve difficulty waiting for rewards (although note that problems with delay of gratification are only one potential aspect of impulsivity). Again, this is highly consistent with theorizing about ADHD as an overall pathology of reward (Luman et al., 2010), and recent findings in the clinical literature about the effects of stimulants in individuals with ADHD (Addicott et al., 2019). Thus, these findings have bearing on our understanding of psychiatric disorders that relate to reward functions.

A second major implication of this review is that changes in DA appear unlikely to linearly relate to any given reward function. Simplistically, if DA is important for reward functioning, then increasing DA should increase that function and decreasing it should decrease that function, as our general hypothesis stated. The studies reviewed here indicate that this is often not the case. In some areas, the effect appeared asymmetrical. For example, antagonists appeared to more consistently affect learning than agonists. In other areas, agonists and antagonists had converging effects. For example, on measures of risk-related decision-making, both increasing and decreasing DA decreased the impact of loss, and reduced the impact of probability on decision-making. Finally, throughout this review, analyses that examined individual differences showed potential U-shaped patterns and modulation of drug effects by baseline functioning. Our findings suggest that both basic and clinical studies must take into consideration the possibility of nonlinear relationships between drugs, DA levels, and performance. For example, recent neuroimaging evidence shows negative correlations between DA levels and reward pathology in addiction, yet positive correlations between DA and reward pathology in ADHD, consistent with this synthesis (Castrellon et al., 2019).

A third major implication is that our ability to detect the effects of DAergic drugs may depend in part on whether the outcome measures are self-report, behavioral, or imaging measures. DAergic drug manipulations appeared to affect neuroimaging measures more consistently than subjective or behavioral measures. For example, DA drugs had little effect on subjective anticipation of upcoming rewards, but produced strong effects on the striatum during anticipation of rewards. Similarly, DA drugs had little effect on self-reported pleasure, but at least some studies indicate that they did affect neural reactivity during receipt of reward. Quite a number of studies indicate that reducing DA function consistently reduces behavioral reward learning as well as brain activity in the ventral striatum and anterior cingulate. Yet, studies that used computational modeling have reported little effect of DA drugs on model-derived learning rate. However, these findings also raise interesting questions about the meaning of brain responses to an event that are not related to behavior or self-report. Ultimately, although imaging appears most sensitive to DA, information about self-report, behavioral and imaging studies will need to be integrated to understand reward processing deficits, and develop medications for associated psychiatric disorders.

B. Recommendations

There are several recommendations that arise from this review. First, there is an urgent need for dose-response studies examining the relationship between DA and reward functioning in humans. As seen in this review, results with single doses can be interpreted post-hoc in multiple ways, and researchers should be cognizant of the possibility that responses are not linearly related to dose. This can only be resolved by using multiple doses to establish a dose-response curve. Second, there is a need to study multiple drugs that manipulate the DA system in the same measures and participants. Any single drug is likely to have actions on several other receptor systems, and so does not provide a pure effect of increasing or decreasing DA function. Converging findings with several drugs with shared actions on the same measures would provide more confidence in results. Further, studies comparing multiple drugs in the same participants and measures would strongly assist our ability to evaluate interpretations that depend on theorizing about differences in the actions or affinities of individual drugs. Third, there is a need to utilize multiple outcome measures to characterize reward function, and examine multiple reward-related processes in the same studies. As we have shown here, outcomes depend strongly on the measures used. Further, although this review strongly suggests different sensitivities to DA across different functions, there are few studies directly and explicitly comparing effects on multiple reward functions for the same doses in the same participants. These recommendations reflect the complexity of measuring the role of DA in studies of reward function.

A further recommendation is that researchers be cognizant of possible baseline differences in DA function, which may affect responses to drugs. For example, variation in baseline dopamine levels could make findings especially difficult to interpret when the dose response is hypothesized to be an inverted-U shape. One approach to addressing this issue is to measure baseline dopamine levels using PET (Westbrook et al., 2020). However, PET studies are difficult and expensive. An alternative approach is to examine the effects of the drug in relation to individual differences in “baseline functioning” assessed behaviorally in the lab. Here we strongly recommend standardizing the behavioral measures that are used to assess baseline functioning, and tentatively recommend working memory capacity as a rough proxy for baseline DA function. Even though other neurotransmitters are involved in working memory, this measure has some evidence for relationships to DA, and already is in more widespread usage than any other, assisting comparability (Cools et al., 2008).

A third recommendation concerns reward type. The reward most commonly used in human studies is money. Whether these findings would generalize to other reward types, such as food, sex, emotional, or social reward is largely unknown. For example, there was some evidence that pleasure from food reward was more strongly influenced by DA function, but there were no studies directly comparing this reward type with another. Only one study in this review compared DA effects on more than one reward in the same participants, contrasting positive music and money (Ferreri et al., 2019). Future studies should compare responses across rewards to determine generalizability of the findings, and to permit better translation to studies with laboratory animals, which generally use food rewards. While there is evidence of a common neurobiology for the processing of all reward types (Volkow et al., 2013), it seems possible base don our findings that DA challenges may have different effects on different types of rewards, so more studies of this kind are needed.

Fourth, drugs that reduce DA function (especially receptor antagonists) appear to have more pronounced effects than drugs that enhance DA function (precursors, reuptake inhibitors, or agonists). This may be because healthy adults are likely to already have close to optimal DAergic functioning and optimal performance, making it difficult to detect improvements. Therefore, reducing DA levels in healthy adults might be a more powerful way to study the effects of DA on reward-related functions. Notably, the only function on which DA antagonists had little effect was pleasure, consistent with the hypothesis that DA is less involved in this process. Therefore, reduction strategies may be more appropriate when the sample involves healthy controls.

Finally, while only healthy human studies were presented here, the studies reviewed set the stage for future translational research. An important advantage of drug challenge studies is that some methods can be performed in both humans and laboratory animals. Future work should aim a-priori to employ similar pharmacological manipulations across laboratory animals, healthy humans, and patients to fully understand reward processing and the ways it can go awry (e.g. Pergadia et al., 2014). In addition to similar pharmacological manipulations, future work should look to utilizing tasks that better translate preclinical to human studies. For example, the Probabilistic Reward Task has been useful in measuring reward responsiveness across species and appears sensitive to DA manipulations (Der-Avakian et al., 2013; Pergadia et al., 2014). Development and use of similar tasks for other reward functions is critical to bridge the gap between basic preclinical research and the role of DA in human disorders.

C. Conclusions

The goal of this review was to summarize what can be learned about the role of DA in reward functioning by looking at drug challenge studies in healthy human subjects. Drug challenge studies have good experimental control, utilize tasks that translate well to animal behavioral studies, and can be combined with neuroimaging. Drug challenge studies in healthy humans reduce confounds that are present when studying psychiatric populations and can therefore provide stronger causal evidence for how DA mediates reward functioning. Here we found that the body of evidence suggests different sensitivity to DA manipulations across different phases of reward processing, non-linear relationships between DA and reward functioning, and different sensitivity of different measures to these effects. Results from these reviewed studies are critical for understanding how deficits in reward functioning are related to psychiatric disorders and for guiding medication development for disorders where anticipation of reward, evaluation of effort or gambling costs, or reward learning is altered, such as depression, substance use, and schizophrenia (Keiflin and Janak, 2015; Smoski et al., 2009; Strauss et al., 2011; Treadway et al., 2012).

Table 1.

Anticipation

Subjective Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Sharot	2009	L-DOPA	100mg	N Drug = 29 N Placebo = 32	Anticipated pleasure of vacation destination	Increase
Dean	2016	bupropion	150mg/day	N Within = 17	Wanting to cues of pleasant & unpleasant	No effect	Sub-acute dosing over 7 days
Knutson	2004	d-amphetamine	0.25mg/kg	N Within = 8	Excitement to cues of monetary reward/loss	Trend toward “flattening” -decrease to reward cues; Increase to loss cues
Dopamine Reduction
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Bjork	2014	APTD	--	N Within = 16	Excitement to cues of monetary reward	No effect
Behavioral Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Ferreri	2019	L-DOPA	100mg	N Within = 27	Skin conductance during anticipation of monetary reward	Increase	Compared to risperidone
Behavioral Findings
Dopamine Reduction
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Ferreri	2019	risperidone	2mg	N Within = 27	Skin conductance during anticipation of monetary reward	Decrease	Compared to L-DOPA
Neuroimaging Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Dean	2016	bupropion	150mg/day	N Within = 17	BOLD during anticipation of pleasant & unpleasant taste	Increase in caudate during both pleasant & unpleasant taste anticipation; Increase during pleasant taste anticipation in pregenual ACC/vmPFC & lateral OFC	No effect on self-report; sub-acute dosing over 7 days
Evers	2017	methylphenidate	40mg	N Within = 24	BOLD during during anticipation of monetary reward/loss	Increase in ventral striatum during reward anticipation
Funayama	2014	modafinil	200mg	N Within = 20	BOLD during anticipation of monetary reward/loss	Increase in bilateral NAc, left middle frontal & precentral gyri, & left cuneus during reward anticipation
Ikeda	2019	bupropion	150mg	N Within = 15	BOLD during anticipation of monetary reward/loss	Increase in right NAc
Kirsch	2006	bromocriptine	1.5mg	N Within = 24	BOLD during anticipation of monetary reward/no reward	Increase in right NAcc	Strongest in individuals with the A1A1 DRD2 genotype - consistent with Inverted-U
O’Daly	2014	d-amphetamine	20mg, 4 times	N Drug = 11 N Placebo = 11	BOLD during anticipation during anticipation of monetary reward/loss	Increase in right caudate during reward anticipation	“Sensitization” paradigm with repeated dose of amphetamine
Curley	2013	benzylpiperazine	200mg	N Within = 13	BOLD during anticipation of monetary reward/loss	No effect in striatum; Decrease in inferior frontal gyrus, insula, mid-occipital regions	Participants did not know if they were anticipating reward or loss
Graf	2016	bupropion	150mg	N Within = 17	BOLD during anticipation of monetary reward/loss	No effect in NAcc
Suzuki	2019	atomoxetine	40mg	N Within = 14	BOLD during anticipation of monetary reward/no reward	No effect in NAcc
Weis	2012	L-DOPA	100mg	N Drug = 27 N Placebo = 27	BOLD during anticipation of monetary reward/no reward	No effect in striatum; Increase in left auditory cortex, left inferior frontal gyrus, anterior cingulate cortex	Auditory adaptation of MID
Wittman	2015	L-DOPA	100mg	N Drug = 14 N Placebo = 14	BOLD during anticipation of monetary reward/loss	No effect in striatum during reward anticipation; Increase in striatum during loss anticipation
Apitz	2014	L-DOPA	150mg	N Drug = 20 N Placebo = 18	ERFs & beta oscillatory power (MEG) during cues of high/low monetary reward	Decrease in impact of reward probability on ERF magnitude, Decrease in beta power
Ivanov	2014	methylphenidate	0.5mg/kg	N Within = 16	BOLD during anticipation of monetary reward/no reward	Decrease in left insula, left caudate, left thalamus, & right cerebellum
Knutson	2004	d-amphetamine	0.25mg/kg	N Within = 8	BOLD during anticipation of monetary reward/loss	Decrease in VS during anticipation, but prolonged activation of VS during reward anticipation; Increase in VS during loss anticipation
Guitart-Masip	2012	L-DOPA	150mg	N Drug = 16 N Placebo = 20	BOLD during orthogonalized go/no-go task	Increase in striatum & SN/VTA	“Go-to-win” paradigm
Dopamine Reduction
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Abler	2007	olanzapine	5mg	N Within = 8	BOLD during cues of monetary reward/no reward	Decrease in VS, anterior cingulate & inferior frontal cortex
Bjork	2014	APTD	--	N Within = 16	BOLD during cues of monetary reward/no reward	Decrease in NAcc
Fiore	2018	risperidone	0.5mg	N Within = 20	BOLD during cues of monetary reward	Decrease in dorsal striatum and right orbitofrontal cortex
Nagano-Saito	2012	APTD	--	N Within = 17	BOLD during cues of monetary reward/no reward	Decrease in areas associated with mesocortical system including NAcc
Schutte	2017	haloperidol	2mg	N Within = 23	ERPs (RRP, PRP, & P300) to cue during go/no-go task	RRP and P300 reduced; no effect on PRP	More evident in those with higher eye blink rate (proxy of higher DA function), consistent with Inverted-U
da Silva Alves	2010	AMPT	1.5g	N Within = 10	BOLD during cues of monetary reward/no reward	No effect in striatum, Increase in left cingulate gyrus
Ye	2011	pramipexole	0.5mg	N Within = 16	BOLD during cues of monetary reward/no reward	Increase in right NAcc; Weakening of interaction between NAcc & prefrontal cortex

Table 2.

Evaluation

Time vs. Reward
Behavioral Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
de Wit	2002	d-amphetamine	10 & 20mg	N Within = 36	Delay discounting	Decrease in traditional task, no effect in “experiential” task	“Experiential” task allowed participants to experience small rewards/short time delays in lab
Kayser	2012	tolcapone	200mg	N Within = 23	Delay discounting	Decrease	Greater effects in individuals with high baseline impulsivity, consistent with Inverted-U
Acheson	2008	d-amphetamine/bupropion	20/150mg & 300mg	N Within = 32	Delay discounting	No effect	Included smokers - smoking status did not affect responses
Soutschek	2020	PF-06412562	6mg, 15mg, 30mg	N Placebo = 30, N Drug (each dose) = 30	Delay discounting	No effect
Pine	2010	L-DOPA	150mg	N Within = 13	Delay discounting	Increase
Dopamine Reduction
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Kelm	2013	APTD	--	N Within = 15	Delay discounting	Poor get poorer or Inverted-U	Increase in COMT gene val/val carriers, decrease in met carriers - possible conflicting effects of COMT in different brain regions complicate interpretation
Hamidovic	2008	pramipexole	0.25mg or 0.50mg	N Within = 10	Delay discounting	No effect
Pine	2010	haloperidol	1.5mg	N Within = 13	Delay discounting	No effect
Arrondo	2015	metoclopramide	10mg	N Within = 14	Delay discounting	Decrease	Delay confounded with probability - computational modeling indicated change in delay parameters
Soutschek	2017	amisulpride	400mg	N Within = 55	Delay discounting	Decrease	Only in lower BMI subjects
Weber	2016	amisulpride	400mg	N Drug = 41 N Placebo = 40	Delay discounting	Decrease
fMRI Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Kayser	2012	tolcapone	200mg	N Within = 23	BOLD during delay discounting	Increase in striatum, pregenual cingulate & anterior insula, dorsolateral & medial frontal cortex	In context of decrease in delay discounting behavior
Pine	2010	L-DOPA	150mg	N Within = 13	BOLD during delay discounting	Increase in striatum, insula, subgenual cingulate, & lateral OFC; Decrease in caudate, insula, & lateral inferior frontal regions	In context of increase in delay discounting behavior
Dopamine Reduction
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Pine	2010	haloperidol	1.5mg	13	BOLD during delay discounting	No effect
Arrondo	2015	metoclopramide	10mg	N Within = 14	BOLD during delay discounting	Decrease in post-central gyms
Risk vs. Reward
Behavioral Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Rigoli	2016	L-DOPA	150mg	N Within = 32	Gambling	Increase	Only in lower BMI subjects
Rutledge	2015	L-DOPA	150mg	N Within = 30	Gambling	Increase
Acheson	2008	d-amphetamine/bupropion	20mg/150mg & 300mg	N Within = 32	Probability discounting/ BART pumps	No effect	Included smokers - smoking status did not affect responses
de Wit	2002	d-amphetamine	10&20mg	N Within = 36	Probability discounting	No effect
Evers	2017	methylphenidate	40mg	N Within = 24	Gambling	No effect
Symmonds	2013	L-DOPA	100mg	N Within = 20 in two studies	Gambling	No effect
White	2007	d-amphetamine	20mg	N Within = 37	BART pumps	No main effect	Increase in males with high reward sensitivity, decrease in males with low reward sensitivity (consistent with “rich get richer”), but no effect in women
Soutschek	2020	PF-06412562	6mg, 15mg, 30mg	N Placebo = 30, N Drug (each dose) = 30	Gambling	Decrease
Campbell-Meiklejohn	2012	methylphenidate	20mg	N Drug = 20 N Placebo = 20	Loss-chasing	Increase for high-stakes bets, decrease for low-stakes bets	Consistent with decreased impact of loss
Norbury	2013	cabergoline	1.5mg	N Within = 20	Gambling	Decrease in “probability distortion”, decreased effect of prior loss	Consistent with decreased impact of loss
Dopamine Reduction
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Arrondo	2015	metoclopramide	10mg	N Within = 14	Probability discounting	No effect	Delay confounded with probability - computational modeling indicated no change in probability parameters
Hamidovic	2008	pramipexole	0.25mg & 0.50mg	N Within = 10	Probability discounting/ BART pumps	No effect
Zack	2007	Haloperidol	3mg	N Within = 18	Gambling	No effect
Burke	2018	amisulpride	400mg	N Drug = 45 N Placebo = 48	Gambling	Overall increase, decrease in “probability distortion”
Campbell-Meiklejohn	2011	pramipexole	0.176mg	N Drug = 20 N Placebo	Loss-chasing	Increase for high-stakes bets, decrease for low-stakes bets	Consistent with decreased impact of loss
Ojala	2018	sulpiride	400mg	N Within = 21	Gambling	No overall effect, decrease in “probability distortion”
Riba	2008	pramipexole	0.5mg	N Within = 15	Gambling	Increase	Only when previous trial resulted in large gain
Effort vs. Reward
Subjective Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Volkow	2004	methylphenidate	20mg	N Within = 16	Motivation for effortful task	Increase
Behavioral Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Soutschek	2020	PF-06412562	6mg, 15mg, 30mg	N Placebo = 30, N Drug (each dose) = 30	Choice of effortful task	Increase
Wardle	2011	d-amphetamine	10 or 20mg	N Within = 17	Choice of effortful task	Increase	Strongest effect under low probability of reward, high effort confounded with time to complete
Westbrook	2020	methylphenidate/suplride	20mg/400mg	N Within = 50	Choice of effortful task	Increase	Greater increase in individuals with lower baseline DA synthesis capacity, consistent with Inverted-U
Zenon	2016	L-DOPA	125mg	N Within = 19	Grip force on effortful task	Increase
Michely	2019	L-DOPA	150mg	N Within = 20	Perseverance in effortful grip task	No change
Dopamine Reduction
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Cawley	2013	APTD	--	N Within = 32	Progressive ratio breakpoint	Decrease	Subjects were women with subsyndromal seasonal affective disorder
Venugopalan	2011	APTD	--	N Within = 47	Progressive ratio breakpoint	Decrease	Participants had varying levels of nicotine use
Michely	2019	haloperidol	1.5mg	N Within = 20	Perseverance in effortful grip task	Decreased impact of reward magnitude
fMRI Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Volkow	2004	methylphenidate	20mg	N Within = 16	Striatal DA release under methylphenidate	Correlated with increased self-reported motivation

Table 3.

Execution

“Simple Invigoration” Theory
Subjective Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Funayama	2014	modafinil	20mg	N Within = 20	Exertion of effort	Increase
Ikeda	2019	bupropion	150mg	N Within = 15	Exertion of effort	No effect
Behavioral Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Michely	2019	L-DOPA	150mg	N Within = 20	Grip strength in rewarded task	Increase
Apitz	2014	L-DOPA	150mg	N Drug = 20 N Placebo = 18	Speed in rewarded task	No effect
Dean	2016	bupropion	150mg/day	N Within = 17	Speed in rewarded task	No effect	Sub-acute dosing over 7 days
Funayama	2014	modafinil	20mg	N Within = 20	Speed in rewarded task	No effect
Graf	2016	bupropion	150mg	N Within = 17	Speed in rewarded task	No effect
Ivanov	2014	methylphenidate	5mg/kg	N Within = 16	Speed in rewarded task	No effect
Mayo	2019	methamphetamine	20mg	N Within = 73	Speed in rewarded task	No effect
Weis	2012	L-DOPA	100mg	N Drug = 27 N Placebo = 28	Speed in rewarded task, conditioning paradigm	No effect
Wittman	2015	L-DOPA	100mg	N Drug = 14	Speed in rewarded task	No effect
Kirsch	2006	bromocriptine	1.5mg	N Within = 24	Speed in rewarded task	Inverted-U	Increase in individuals with the A1A1 DRD2 genotype, decrease for other genotypes, consistent with Inverted-U
Dopamine Reduction
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Abler	2007	olanzapine	5mg	N Within = 8	Speed in rewarded task	Decrease
Bjork	2014	APTD	--	N Within = 16	Speed in rewarded task	No effect
da Silva Alves	2014	AMPT	--	N Within = 10	Speed in rewarded task	No effect
Fiore	2018	risperidone	0.5mg	N Within = 20	Time to reach required grip strength in rewarded task	No effect
Michely	2019	haloperidol	1.5mg	N Within = 20	Grip strength in rewarded task	No effect
Nagano-Saito	2012	APTD	--	N Within = 17	Speed in rewarded task	No effect
Schutte	2017	haloperidol	2mg	N Within = 23	Speed in rewarded task	No effect
Neuroimaging Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Dean	2016	bupropion	150mg/day for 7 days	N Within = 17	BOLD during execution of hard or easy task to gain pleasant or avoid unpleasant task	Increase in caudate, vmPFC, dACC/paracingulate gyrus & putamen	Contrast was between easy/pleasant & hard/pleasant trials; sub-acute dosing over 7 days
Dopamine Reduction
Fiore	2018	risperidone	0.5mg	N Within = 20	BOLD during execution of grip to gain high vs. low reward	Decrease in external globus pallidus to high reward, increase in substantia nigra to low reward
“Average Rate of Reward” Theory
Behavioral Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Beierholm	2013	L-DOPA	150mg	N Drug = 30 N Placebo = 30	Computational model of influence of average rate of reward	Increase
Zenon	2016	L-DOPA	125mg	N Within = 20	Grip force on effortful task, when force was related to time to complete task	No effect
“Go to Win” Theory
Behavioral Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Guitart-Masip	2012	L-DOPA	150mg	N Drug = 16 N Placebo = 20	Speed on orthogonalized go/no-go task	Increase for all go trials, regardless of win/loss type
Swart	2017	methylphenidate	20mg	N Within = 99	Speed on orthogonalized go/no-go task	No effect	Task required contingencies learning
Dopamine Reduction
First Author	Year	Drug	Dose	N	Measure(s)	Result(s)	Notes
Weber	2016	amisulpride	400mg	N Drug = 41 N Placebo = 40	Pavlovian-to-instrumental transfer	Decrease

Table 4.

Pleasure

Subjective Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Dean	2016	bupropion	150mg	N Within = 17	Liking ratings of food	No effect	Sub-acute dosing over 7 days
Knutson	2004	d-amphetamine	0.25mg/kg	N Within = 8	Excitement ratings to small vs. large monetary reward	“Flattened”
Rutledge	2015	L-DOPA	150mg	N Within = 30	Happiness ratings to small vs. large monetary reward	“Flattened”
Wardle	2012	d-amphetamine	10mg & 20mg	N Within = 36	Positive ratings of pictures	Increase/’Flattened”	Enhanced positive ratings of both pleasant & unpleasant pictures
Ferreri	2018	L-DOPA	100mg	N Within = 27	Liking ratings of music	Increase	Compared to risperidone
Dopamine Reduction
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Bjork	2014	APTD	--	N Within = 16	POMS (total mood disturbance) to monetary reward	No effect
Brauer	1995	pimozide	4mg	N Within = 12	POMS (elated, positive mood) ARCI (A, MBG) to d-amphetamine	No effect
Brauer	1995	fluphenazine	3mg & 6mg	N Within = 12	POMS (elated, positive mood) ARCI (A, MBG) to d-amphetamine	No effect
Brom	2016	haloperidol	3mg	N Drug = 29 N Placebo = 19	Affective ratings of sexual pictures	No effect
Frank	2016	APTD	--	N Within = 34	Liking ratings of food	No effect
Hardman	2012	APTD	--	N Within = 17	Liking ratings of food	No effect
Leyton	2007	APTD	0.3mg	N Within = 14	POMS (elated) VAS (euphoria) to d-amphetamine	No effect
Barrett	2008	APTD	--	N Within = 18	VAS (euphoric) to alcohol	No effect
Leyton	2005	APTD	--	N Within = 8	VAS (euphoric) to cocaine	No effect
Venugopalan	2011	APTD	--	N Within = 47	VAS (euphoria) to nicotine	No effect	Participants had varying levels of nicotine use
McCabe	2011	sulpiride	400mg	N Drug = 15 N Placebo = 15	Liking ratings of mood	No effect
McCabe	2012	pramipexole	.2 mg	N Within = 16	Liking ratings of food POMS (elated, positive mood)	No effect
Wachtel	2002	haloperidol	3mg	N Within = 18	VAS (euphoria) ACRI (A, GB) to methamphetamine	No effect
Zack	2007	haloperidol	3mg	N Within = 18	VAS (enjoyment, excitement) to gambling	No effect
Ferreri	2018	risperidone	2mg	N Within = 27	Liking ratings of music	Decrease	Compared to L-DOPA
Behavioral Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Ferreri	2018	L-DOPA	100mg	N Within = 27	EDA to music	Increase	Compared to risperidone
Wardle	2012	d-amphetamine	10mg & 20mg	36 N Within = 36	EMG to pictures	Increase for zygomatic; Decrease for corrugator	Zygomatic is “smile”, corrugator is “frown”
Behavioral Findings
Dopamine Reduction
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Ferreri	2018	risperidone	2mg	N Within = 27	EDA to music	Decrease	Compared to L-DOPA
Neuroimaging Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Knutson	2004	d-amphetamine	.25mg/kg	N Within = 8	BOLD to monetary reward	No effect
Dean	2016	bupropion	150mg	N Within = 17	BOLD to food	“Flattened”- Increase in medial OFC to pleasant & unpleasant; Increase in amygdala & VS to unpleasant; Decrease in caudate to pleasant	No effect on self-report; sub-acute dosing over 7 days
Melrose	2016	d-amphetamine	.5mg/kg	N Within = 11	BOLD to sucrose	“Flattened” - Increase in right middle insula to sucrose; Decrease in dorsal ACC to sucrose
Abler	2011	bupropion	150mg	N Within = 18	BOLD to erotic video	Increase in posterior midcingulate, mediodorsal thalamus, amygdala	Not related to subjective sexual functioning; sub-acute dosing over 7 days
Abler	2012	bupropion	150mg	N Within = 18	BOLD to erotic video	Increase in NAcc; decreased interaction between NAcc and avPFC	Only compared to paroxetine
O’Daly	2014	d-amphetamine	2 mg, 4 times	N Drug = 11 N Placebo = 11	BOLD to monetary reward	Increase in amygdala	“Sensitization” paradigm with repeated doses of amphetamine
Oei	2012	L-DOPA	100mg	N Drug = 18 N Placebo = 17	BOLD to erotic images	Increase in NAcc & dorsal ACC
Ivanov	2014	methylphenidate	.5mg/kg	N Within = 16	BOLD to monetary reward	Decrease in mid-occipital & left inferior parietal cortex; Decrease in left insula & pallidum
Dopamine Reduction
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Bjork	2014	APTD	--	N Within = 16	BOLD to monetary reward in MID	No effect	No effect on self-report
Sescousse	2016	sulpiride	400mg	N Within = 22	BOLD to near-misses in a gambling task	No effect
Frank	2016	APTD	--	N Within = 34	BOLD to food	Decrease in striatum; Increase in superior frontal gyrus	No effect on self-report
McCabe	2011	sulpiride	400mg	N Drug = 15 N Placebo = 15	BOLD to food	Decrease in VS & ACC	No effect on self-report
McCabe	2012	pramipexole	0.25mg	N Within = 16	BOLD to food	Decrease in VS & dorsal ACC to pleasant food; Decrease in inferior OFC & insula to unpleasant food	No effect on self-report
Oei	2012	haloperidol	3mg	N Drug = 18 N Placebo	BOLD to erotic images	Decrease in NAcc & dorsal ACC
Riba	2008	pramipexole	0.5mg	N Within = 15	BOLD to monetary reward in gambling task	Decrease in rostral basal ganglia, including VS
Graf	2015	amisulpride	200mg/d	N Within = 19	BOLD to erotic video	No effect
Ye	2011	pramipexole	0.5mg	N Within = 16	BOLD to monetary reward in MID	Increase in right NAcc

Table 5.

Reward Learning

“Raw” Behavioral Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Pessiglione	2006	L-DOPA	100mg	N Drug = 13 N Placebo = 13	Money won & % of rewarding choices on instrumental learning task	Increase	L-DOPA only different from haloperidol, not placebo
Pleger	2009	L-DOPA	100mg	N Drug = 10 N Placebo = 10	Accuracy on a somatosensory judgement task	Increase
Bellebaum	2016	modafinil	200mg	N Drug = 18 N Placebo = 21	% correct choice on probabilistic selection task during learning phase	No effect
Bernacer	2013	methamphetamine	0.3mg/kg	N Within = 18	Accuracy on reinforcement learning task	No effect
Bernacer	2013	methamphetamine + amisulpride	0.3mg/kg 400 mg	N Within = 18	Accuracy on reinforcement learning task	No effect
Diederen	2017	bromocriptine	2.5mg	N Drug = 20 N Placebo = 21	Accuracy in predicting reward magnitude based on reward history	No effect	Reduced accuracy at a trend level
Dodds	2008	methylphenidate	60mg	N Within = 17	# of errors preceding a switch & probability of switching after error feedback	No effect
Jocham	2011	amisulpride	200mg	N Within = 16	% correct choice on probabilistic selection task during learning phase	No effect
Weis	2013	L-DOPA	100mg	N Drug = 27 N Placebo = 28	Learning rate on instrumental conditioning paradigm	No effect
van der Schaff	2012	bromocriptine	1.25mg	N Within = 22	Proportion of correct responses following unexpected outcomes on reversal learning task	Decrease	Compared to sulpiride, not placebo
Vo	2006	L-DOPA	100mg	N Within = 26	Accuracy on reversal learning task	Decrease
Clatworthy	2009	methylphenidate	60mg	N Within = 10	Accuracy on reversal learning task	Inverted-U	Increased learning in those with small displacements & impaired learning in those with larger displacements in caudate nucleus
Cools	2009	bromocriptine	1.25mg	N Within = 11	Relative reversal learning scores on reversal learning task	Inverted-U	Improved learning in those with lower dopamine synthesis in the striatum measured by PET
Mueller	2014	sulpiride	200mg	N Drug = 43 N Placebo = 43	% of catch trials on ball catching task	Inverted-U	Worse task performance in extraverts compared to introverts
Cohen	2007	cabergoline	1.5mg	N Within = 22	Speed to reach learning criteria on reversal learning task	Rich get richer	Sped up learning in A- (increased DA) genotype; slowed learning in A+ (reduced DA) genotype
van der Schaff	2013	methylphenidate	20mg	N Within = 19	Proportion of correct responses following unexpected outcomes on reversal learning task	Rich get richer	Increased learning in greater working memory & impaired in low working memory
Frank	2006	haloperidol	2mg	N Within = 28	Accuracy to positive stimulus & accuracy in withholding response to most negative stimulus	Inverted-U	“Go-to-win” paradigm; haloperidol increased go learning & decreased no-go in low working memory
Swart	2017	methylphenidate	20mg	N Within = 99	Probability of correct choice for go to win	No effect	“Go-to-win” paradigm
Guitart-Masip	2014	L-DOPA	150mg	N Drug = 30 N Placebo = 29	Probability of correct choice for go & correct omission for no-go	Decrease	“Go-to-win” paradigm
Dopamine Reduction
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Diederen	2017	sulpiride	600mg	N Drug = 22 N Placebo = 21	Accuracy in predicting reward magnitude based on reward history	Decrease
Forster	2017	haloperidol	3mg	N Drug = 18 N Placebo = 21	Accuracy on a time estimation task	Decrease	Sample from Zirnheld et al. 2004
Janssen	2015	sulpiride	400mg	22	Accuracy on reversal trials of reversal learning task	Decrease
Jocham	2014	amisulpride	400mg	N Within = 22	Probability of selecting rewarding option on instrumental learning task	Decrease	Interaction was marginally significant & results reflect a post hoc test
Jocham	2014	memantine	20mg	N Within = 22	Probability of selecting rewarding option on instrumental learning task	Decrease	Approach learning was decreased, avoidance learning was unaffected
Leyton	2007	APTD	--	N Within = 13	Accuracy on rewarding-correct trials on go/no-go task	Decrease
Pessiglione	2006	haloperidol	1mg	N Drug = 13 N Placebo = 13	Money won on instrumental learning task	Decrease	Haloperidol group won less money compared to L-DOPA, not placebo
Pizzagali	2008	pramipexole	5mg	N Drug = 11 N Placebo = 13	Reward bias over blocks & win-stay strategy on probabilistic reward task	Decrease
Pleger	2009	haloperidol	2mg	N Drug = 10 N Placebo = 10	Accuracy on a somatosensory judgement task	Decrease
Tremblay	2011	haloperidol	3mg	N Within = 20	Relationship between payoff & bet size on slot machine gambling task	Decrease
Zirnheld	2004	haloperidol	3mg	N Drug = 17 N Placebo = 18	Accuracy on a time estimation task	Decrease
Brom	2016	haloperidol	3mg	58	Conditioned sexual arousal	No effect
de Wit	2012	APTD	--	N Drug = 14 N Placebo = 14	Accuracy on instrumental learning	No effect
Dodds	2008	sulpiride	400mg	N Within = 17	# of consecutive errors preceding a switch & probability of switching after error feedback	No effect
Eisenegger	2014	sulpiride	800mg	N Drug = 41 N Placebo = 35	Probability of choosing the correct symbol	No effect
Hamidovic	2008	pramipexole	.025 & 0.5mg	N Within = 10	Errors of omission & commission on go/no-go & performance on card preservation task	No effect
Robinson	2010	APTD	--	N Within = 27	Proportion of errors for each type of trial on reversal learning	No effect
Tobia	2014	APTD	--	N Drug = 25 N Placebo = 30	Reaching most lucrative state in strategic sequential investment	No effect
Frank	2006	cabergoline	1.25mg	N Within = 28	Accuracy when most positive stimulus is presented & accuracy in withholding response to most negative stimulus	Inverted-U	Increased avoidance learning relative to approach learning in high working memory
van der Schaff	2012	sulpiride	400mg	N Within = 22	Proportion of correct responses following unexpected outcomes on reversal learning task	Inverted-U	Greater working memory associated with greater sulpiride-induced increases in learning
Computational Modeling Findings
Dopamine Enhancement
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Diederen	2017	bromocriptine	2.5mg	N Drug = 20 N Placebo = 21	Model fit across several computational models; α, PE scaling & decay in α as predicted by an adaptive Pearce-Hall computational model	No effect on α or β No effect on PE scaling No effect on decay
Guitart-Masip	2014	L-DOPA	150mg	N Drug = 30 N Placebo = 29	α, β, Pavlovian & bias parameters as predicted by modified Rescorla-Wagner computational models	No effect on α or β Decrease in Pavlovian parameter No effect on bias parameter
Jocham	2011	amisulpride	200mg	N Within = 16	α & β as predicted by standard Q-leaning computational model	No effect on α or β
Wunderluch	2012	L-DOPA	150mg	N Within = 18	α, β, & model-based & model-free parameters on hybrid model	No effect on α or β
Cools	2009	bromocriptine	1.25mg	N Within = 11	α predicted by an updated Q-learning model that included a rule to update the alternative choice in the opposite direction	Inverted-U	Increased α in low DA synthesis, decreased α in high DA synthesis; similar results found with working memory
Bernacer	2013	methamphetamine alone & methamphetamine + amisulpride	0.03mg/kg & 0.3mg/kg + 400mg	N Within = 18	α & β as predicted by standard Q-learning computational model	Increase in α No effect on β
Dopamine Reduction
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Diederen	2017	sulpiride	600mg	N Drug = 22 N Placebo = 21	α, PE scaling & decay in α as predicted by an adaptive Pearce-Hall computational model	No effect on α or β No effect on decay Decrease in scaling parameter
Eisenegger	2014	sulpiride	800mg	N Drug = 41 N Placebo = 35	α & β as predicted by standard Q-learning computational model	No effect on α Increased β	Only in A1 carriers of the TaqlA polymorphism
Jocham	2014	amisulpride	400mg	N Within = 22	Model fit; α, β, & performance as predicted by updated (relative) Q-learning computational model	No effect on α or β Decreased performance
Jocham	2014	memantine	20mg	N Within = 22	Model fit; α, β, & performance as predicted by updated (relative) Q-learning computational model	No effect on α or β No effect on performance
Tobia	2014	APTD	--	N Drug = 25 N Placebo = 30	Model fit, α, & β predicted by standard Q-learning & fictive prediction error (FPE) models on strategic sequential investment task	No effect on α No effect on β
Neuroimaging Findings - fMRI
Dopamine Enhancement
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Dodds	2008	methylphenidate	60mg	N Within = 17	BOLD to switch after negative & stay after positive feedback	Increase in ACC & PFC during positive feedback; Decrease in VS during switch
Jocham	2011	amisulpride	200mg	N Within = 16	BOLD correlates of prediction errors	Increase in VS & frontal cortex	vmPFC correlated with overall learning under amisulpride
Pessiglione	2006	L-DOPA	100mg	N Drug = 13 N Placebo = 13	BOLD correlates of prediction errors	Increase in VS	VS activity correlated with RPE
Pleger	2009	L-DOPA	100mg	N Drug = 10 N Placebo = 10	BOLD to monetary reward	Increase in VS & OFC
Diederen	2017	bromocriptine	2.5mg	N Drug = 20 N Placebo = 21	BOLD correlates of prediction errors	No effect
van der Schaff	2012	bromocriptine	1.25mg	N Within = 22	BOLD to rewards & punishments	No effect	Compared to sulpiride, did not differ from placebo
Weis	2013	L-DOPA	100mg	N Drug = 27 N Placebo = 28	BOLD to reward feedback	No effect
Graf	2016	bupropion	150mg	N Within = 17	BOLD to reward outcome on MID task	No effect
Bernacer	2013	methamphetamine alone & methamphetamine + amisulpride	0.03mg/kg & 0.3mg/kg + 400mg	N Within = 18	BOLD correlates of prediction errors	Decrease in left NAcc	Not restored by amisulpride
Evers	2017	methylphenidate	40mg	N Within = 24	BOLD correlates of prediction errors	Decrease in VS
Dopamine Reduction
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
Diederen	2017	sulpiride	600mg	N Drug = 22 N Placebo = 21	BOLD correlates of prediction errors	Decrease in VS & midbrain
Dodds	2008	methylphenidate + sulpiride	60mg 400mg	N Within = 17	BOLD to switch after negative & stay after positive feedback	Decrease in VS during switch; increase in ACC & PFC during positive feedback	Sulpiride did not change methylphenidate effects
Jocham	2014	amisulpride	400mg	N Within = 22	BOLD correlates of prediction errors	Decrease in VS & caudate
Jocham	2014	memantine	20mg	N Within = 22	BOLD correlates of prediction errors	Decrease in VS & caudate
Pessiglione	2006	haloperidol	1mg	N Drug = 13 N Placebo = 13	BOLD correlates of prediction errors	Decrease in striatum
Pleger	2009	haloperidol	2mg	N Drug = 10 N Placebo = 10	BOLD to monetary reward	Decrease in VS & OFC
Tobia	2014	APTD	--	N Drug = 25 N Placebo = 30	BOLD correlates of prediction errors	Decrease in thalamus & amygdala
van der Schaff	2012	sulpiride	400mg	N Within = 22	BOLD to monetary reward	Inverted-U	Sulpiride-induced increases in VS correlated with reversal learning in greater working memory
Neuroimaging Findings - EEG
Dopamine Enhancement
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
de Bruijn	2004	d-amphetamine	15mg	N Within = 12	ERN amplitude to errors on Flanker task	Increase	ERN more negative
Mueller	2011	sulpiride	200mg	N Drug = 90 N Placebo = 79	ERN amplitude to errors on Flanker task	Inverted-U	Sulpride enhanced ERN in Val/Val genotype, reduced ERN in Met carriers
Mueller	2014a	sulpiride	200mg	N Drug = 45 N Placebo = 41	FRN amplitude to errors on ball catching task	Inverted-U	Same participants as Mueller 2014b; Decreased only in extraverts compared to introverts
Mueller	2014b	sulpiride	200mg	N Drug = 45 N Placebo = 38	FRN amplitude to errors on ball catching task	Inverted-U	Same participants as Mueller 2014a; Val carriers had enhanced FRNs; under low dose sulpiride this effect was reversed
Apitz	2014	L-DOPA	150mg	N Drug = 20 N Placebo = 18	ERFs to monetary reward	No effect	ERFs potentially represent the MEG counterpart to the FRN
Dopamine Reduction
First Author	Year	Drug	Dose	n	Measure(s)	Result(s)	Notes
de Bruijn	2006	haloperidol	2.5mg	N Within = 14	ERN amplitude to errors on Flanker task	Decrease	ERN was attenuated (less negative)
Santesso	2009	pramipexole	5mg	N Drug = 7 N Placebo = 13	FRN to correct feedback	Decrease	Subset of participants from Pizzagali 2008
Zirnheld	2004	haloperidol	3mg	N Drug = 17 N Placebo = 18	ERN amplitude to errors on Flanker task	Decrease
Forster	2017	haloperidol	3mg	N Drug = 18 N Placebo = 21	FRN amplitude to negative feedback on a temporal interval learning task	No effect	Same participants from Zirnheld et al. 2004
Larson	2015	APTD	--	N Within = 12	ERN amplitude on Stroop task	No effect

Highlights.

• The role of dopamine in human reward functioning is highly researched

• Drug challenge studies are translational and have good experimental control

• Dopamine drug challenge studies in healthy adults were summarized by reward phase

• Dopamine drugs have differing effects on various reward phases

• Drug effects are likely nonlinear and behavioral effects depend on outcome measure