Peril and Pleasure: An RDoC-inspired examination of threat responses and reward processing in anxiety and depression

Abstract

As a step toward addressing limitations in the current psychiatric diagnostic system, the NIMH recently developed the Research Domain Criteria (RDoC) to stimulate integrative research—spanning self-report, behavior, neural circuitry, and molecular/genetic mechanisms—on core psychological processes implicated in mental illness. Here, we use the RDoC conceptualization to review research on threat responses, reward processing, and their interaction. The first section of the manuscript highlights the pivotal role of exaggerated threat responses—mediated by circuits connecting the frontal cortex, amygdala, and midbrain—in anxiety, and reviews data indicating that genotypic variation in the serotonin system is associated with hyperactivity in this circuitry, which elevates the risk for anxiety and mood disorders. In the second section, we describe mounting evidence linking anhedonic behavior to deficits in psychological functions that rely heavily on dopamine signaling, especially cost/benefit decision-making and reward learning. The third section covers recent studies that document negative effects of acute threats and chronic stress on reward responses in humans. The mechanisms underlying such effects are unclear, but new optogenetic data in rodents indicate that GABAergic inhibition of midbrain dopamine neurons, driven by activation of the habenula, may play a fundamental role in stress-induced anhedonia. In addition to its basic scientific value, a better understanding of interactions between the neural systems that mediate threat and reward responses may offer relief from the burdensome condition of anxious depression.

Introduction

The Diagnostic and Statistical Manual of Mental Disorders^[1] and International Classification of Diseases^[2] provide a valuable common language for clinicians and researchers, but they do not reflect recent advances in our understanding of pathophysiology. To address this limitation, the National Institute of Mental Health (NIMH) recently launched the Research Domain Criteria (RDoC) initiative. Rooted in a dimensional approach to mental health, the RDoC matrix (http://www.nimh.nih.gov/research-priorities/rdoc/nimh-research-domain-criteria-rdoc.shtml#toc_matrix) provides a new framework for psychopathology research. The matrix rows list five systems encompassing broad domains of function—Positive Valence Systems, Negative Valence Systems, Cognitive Systems, Systems for Social Processes, and Arousal/Regulatory Systems—while the columns list units of analysis: genes, molecules, cells, circuits, physiology, behavior, and self-report. Each domain includes several lower-order constructs. For example, Negative Valence Systems comprises potential threat, acute threat, sustained threat, loss, and frustrative non-reward, while Positive Valence Systems includes approach motivation, initial responsiveness to reward, sustained responsiveness to reward, reward learning, and habit. By focusing on core psychological functions and incorporating data from several levels of analysis, the RDoC initiative aims to fundamentally advance our understanding of pathophysiology.

Inspired by the RDoC matrix, we review work on two topics central to anxiety and depression—namely, threat reponses and reward processing. We focus primarily on vigilance for potential and acute threats, and with respect to reward processing, we mainly consider reward anticipation, reward learning, and cost/benefit decision-making. The first two sections summarize the sizable threat and reward literatures, integrating well-established findings across behavioral, circuit, and molecular/genetic levels of analysis. By contrast, the third section highlights novel investigations of the effects of threat and stress on reward processing. This work is improving our understanding of stress-induced anhedonia, and may inform therapeutic interventions for anxiety, depression, and the burdensome condition of anxious depression.

I. Threat Responses and Anxiety

Behavior

Ethology provides a useful entryway into threat research.^[3] Organisms must be wary of predators and conspecifics, and a consistent sequence of threat-related behaviors—vigilance, risk assessment, defense—is observed across mammals.^[4] Anxiety disorders are marked by increased threat vigilance, as documented by a meta-analysis of studies using the emotional Stroop, emotional spatial cueing, and dot-probe tasks that reported a reliable threat-bias in clinically and sub-clinically anxious samples, with no evidence for threat-bias in non-anxious individuals.^[5] These findings converge with self-report data confirming increased concern over uncertain threats in anxious individuals, particularly those with generalized anxiety disorder,^[6] but diverge from results in depressed subjects, who do not show increased threat vigilance but are slow to disengage from sad material, especially if it is personally relevant.^[7–9] Thus, threat vigilance is a stronger indicator of anxiety than depression.

Neural Circuits

Work in non-clinical samples has provided valuable insight into the neural systems that mediate threat signaling. To dissociate brain regions signaling acute threat versus sustained anxiety, one functional magnetic resonance imaging (fMRI) study presented negative and neutral pictures at predictable and unpredictable intervals.^[10] The amygdala showed transient responses to negative pictures that did not depend critically on predictability (Figure 1A), underscoring its role in acute threat signaling, and a similar response profile was seen in the periaqueductal gray (PAG), a midbrain region activated during the experience of negative emotion.^[11] By contrast, the bed nucleus of the stria terminalis (BNST; Figure 1B) signaled sustained anxiety, with no transient response to negative pictures but linearly increased activation across conditions as a function of anxiety (neutral/predictable < neutral/unpredictable < negative/predictable < negative/unpredictable). Of note, the ventral anterior cingulate cortex/ventromedial prefrontal cortex (vACC/VMPFC) showed the opposite response pattern, with stronger activation when anxiety was lowest (neutral/predictable condition) versus when it was highest (negative/unpredictable condition). Furthermore, weak vACC/VMPFC recruitment in unpredictable contexts correlated with greater self-reported anxiety intolerance. Collectively, these data support a role for the vACC/VMPFC in safety signaling,^[12] possibly reflecting its regulatory influence on the amygdala.^[13]

Figure 1.

Neural circuitry implicated in anxiety and threat responses. (A) The amygdala shows a transient response to threat cues. (B) Sustained anxiety is reflected in activation of the BNST. Images A and B reprinted from Sommerville LH, Wagner DD, Wig GS, Moran JM, Whalen PJ, Kelley M. “Interactions between transient and sustained neural signals support the generation and regulation of anxious emotion.” Cerebral Cortex 2013;23:49–60, by permission of Oxford University Press. (C) As a threatening predator moves closer, brain activation shifts from forebrain regions such as the VMPFC (purple) to more posterior regions, prominently including the PAG in the midbrain (white). Image reprinted with permission of the Society for Neuroscience, from Mobbs D, Marchant JL, Hassabis D, Seymour B, Tan G, Gray M, Petrovic P, Dolan RJ, Frith CD. “From threat to fear: the neural organization of defensive fear systems in humans.” Journal of Neuroscience 2009;29:12236–12243; permission conveyed through Copyright Clearance Center, Inc. (D) Together with the amygdala, the insula shows a hyperactive response to threat cues in anxiety disorders. These data depict an exaggerated bilateral insula response to phobia-related versus neutral words in spider phobics. Image reprinted from Neuroscience Letters 372, Straube T, Mentzel HJ, Glauer M, Miltner WHR. “Brain activation to phobia-related words in phobic subjects”, pages 204–208, copyright 2004, with permission from Elsevier. BNST: bed nucleus of the stria terminalis; PAG: periaqueductal gray; VMPFC: ventromedial prefrontal cortex. Reproduced with permission from the publishers.

Many of the same regions have emerged from fMRI studies designed to identify brain regions mediating anxiety responses along a “threat imminence continuum”.^[14,15] In these studies, participants navigated a maze while being chased by a virtual predator that delivered electrical shocks. A forebrain-to-midbrain activation shift was seen as the predator drew near (Figure 1C). When the predator was distant, activation was observed in the basolateral amygdala^[15] and vACC/VMPFC^[14,15]. By contrast, when the predator was imminent and shock delivery was unavoidable (“circa-strike”), activation of the PAG^[14,15], central amygdala^[15], dorsal anterior cingulate cortex (dACC)^[14], and insula^[14] was seen. Dread of capture and decreased escape confidence correlated with increased PAG activation in the circa-strike phase^[15], and this was associated with motor errors that suggested a panic-like response^[14].

The results from these studies^[14,15] indicate that the vACC/VMPFC is engaged during threat assessment and the PAG in active defense, while the dACC circa-strike response may reflect conflict between two response options: fight or flight. The data also highlight functionally dissociable amygdala nuclei: the basolateral nucleus contributes to threat assessment, while the central nucleus initiates defensive behavior through its connections with the midbrain.^[16,17] Overall, these studies map the evolving threat response, which begins in the vACC/VMPFC and basolateral amygdala, runs through the ACC, and finishes in the PAG, insula, and central amygdala.^[14,15]

Anxiety disorders have been consistently linked to aberrant responses in several of these regions. A meta-analysis of fMRI and positron emission tomography (PET) studies of specific phobia, social anxiety disorder, and post-traumatic stress disorder (PTSD) confirmed hyperactive amygdala responses to negative stimuli across these three disorders.^[18] Moreover, decreased activation of the dACC, vACC, and VMPFC was seen and appeared specific to PTSD, possibly contributing to emotional dysregulation such as numbing and flashbacks.^[19,20] Finally, increased midbrain gray matter volume has been observed in panic disorder^[21] and correlated with disorder severity.^[22] Thus, panic disorder may reflect structural or functional abnormalities in the PAG and other regions mediating defensive responses.

Anxiety has also been linked to hyperactivity of the insula, a brain region that responds to interoceptive signals (Figure 1D).^[12,18] For instance, individuals high in anxiety sensitivity worry about becoming anxious and avoid anxiety-provoking stimuli^[23], and several reports describe positive correlations between anxiety sensitivity and indices of insula structure and function.^[24–27] Thus, insula hyperactivation may reflect a proclivity to monitor the internal environment for potential “threats”, such as rapid heartbeat.

Because many anxiety disorders have a childhood onset, ^{[e.g., 28]} considering developmental antecedents is important. Several studies have linked excessive threat reactivity and anxiety to the behaviorally inhibited temperament. Behavioral inhibition refers to extreme caution and timidity upon exposure to novel stimuli, and it can be reliably coded in infancy.^[29] Behaviorally inhibited children are at increased risk for social anxiety disorder in early childhood^[30] and adolescence,^[31] and this appears to be at least partly due to lower activation thresholds in the sympathetic nervous system and hypothalamic-pituitary-adrenal axis.^[29] Strikingly, individuals who were behaviorally inhibited at 2 years old display a stronger bilateral amygdala response to novel faces in adulthood than their uninhibited counterparts.^[32,33] Thus, social anxiety disorder may reflect a long-standing tendency for the amygdala to over-react to novelty, including unfamiliar faces.

Finally, two findings from the fear conditioning literature provide additional evidence of the centrality of learning and memory, and VMPFC/amygdala circuitry, in pathological anxiety. First, anxiety is associated with deficits in discriminative learning and fear extinction, which could exacerbate symptoms and prolong their duration. In discrimination paradigms, one conditioned stimulus (CS+) is consistently paired with an aversive, unconditioned stimulus (US), while a second conditioned stimulus (CS−) is not. Healthy individuals typically develop a fear response to the CS+ but not the CS−. By contrast, several studies report weak discriminative conditioning in anxious individuals driven primarily by excessive responding to the CS−.^[34–36] This may reflect failure to inhibit fear responses during CS− presentation, or overgeneralization of fear responses from the CS+ to the CS−.^[37,38] Second, there are reports of heightened responses to the CS+ during extinction in anxious individuals.^[39,40] It will be important to determine whether this reflects failure to learn that the CS+ no longer predicts US delivery, or failure to use that learning to override the previously formed CS+/US relationship. Finally, maintaining fear extinction depends on memory retrieval processes mediated by the VMPFC,^[41] and this mechanism appears to be dysfunctional in some anxiety disorders.^{[e.g., PTSD; 42]}

Molecules/Genes

Individual differences in threat circuitry responsivity have been linked to variation in the serotonin transporter gene (5-HTTLPR). Compared to long allele homozygotes, 5-HTTLPR short allele carriers showed bilateral amygdala hyperactivation to fearful and angry faces.^[43–45] Furthermore, decreased functional coupling between the amygdala and perigenual cingulate, as well as decreased gray matter volume in both regions, has been observed in short allele carriers.^[46] These findings support hypotheses linking emotional stability to serotonergic functioning,^[47,48] and suggest a genetic contribution to amygdala hypersensitivity in anxiety.

However, caution must be exercised when extrapolating from these studies to conclusions about excessive anxiety. First, the initial demonstrations of amygdala hyperactivation in 5-HTTLPR short allele carriers involved healthy samples displaying normative anxiety,^[43,44] suggesting that neither possession of the short allele nor amygdala hyperactivity is sufficient to yield an anxious phenotype. Second, whether stress can explain links between the 5-HTTLPR short allele and anxiety—or psychopathology more broadly—is unclear. Enthusiasm stems from a well-known report that the relationship between 5-HTTLPR genotype and depressive illness depends on life stress.^[49] However, two meta-analyses did not find support for this gene x environment interaction,^[50,51] and another concluded that most candidate gene x environment interactions, including the 5-HTTLPR x stress interaction, are unreliable,^[52] largely because most studies are underpowered. Neuroimaging may help circumvent this limitation, as neural data lie closer to the genetic effects of interest than self-report data.^{[but see 53]} Along these lines, one study found a positive correlation between life stress and resting activation of the amygdala and hippocampus, but only in 5-HTTLPR short allele carriers.^[54] Finally, it is important not to overlook the environment in gene x environment interactions. 5-HTTLPR short allele carriers appear to be exquisitely sensitive to environmental cues, which engenders anxiety when stressors abound. However, when conditions are more salubrious, 5-HTTLPR short allele carriers may be especially able to take advantage.^[55] For instance, one study^[56] used a gambling task to show that, compared to long allele carriers, 5-HTTLPR short allele were more sensitive to changes in their chances of winning, altering their behavioral adaptively to maximize their gains. Thus, increased responsivity to negative cues in short allele carriers may only be one side of the story—they may be more sensitive to positive cues as well.^[55]

Summary

Heightened vigilance for potential threats is a prominent feature of anxiety that is supported by the BNST, basolateral amygdala, and vACC/VMPFC; other regions, such as the PAG, central amygdala, dACC, and insula, respond more robustly when threats are imminent. Specific anxiety disorders have been associated with hyperactivity in some of these structures (amygdala, insula) and hypoactivity in others (e.g., hypoactivation of dACC, vACC, VMPFC in PTSD). Individual differences in amygdalar responses to potential threat vary with 5-HTTLPR genotype, but this may be counterbalanced by greater sensitivity to positive features of the environment. Overall, the evident connections among genetic, neural, and behavioral systems that support threat vigilance make potential threat an excellent fit for the RDoC initiative.

II. Anhedonia and Reward Processing

While heightened negative affect characterizes both anxiety and depression, anhedonia plays a more central role in depressive illness.^[57,58] Anhedonia research has flourished with the development of a basic literature that describes partially dissociable neural systems for reward anticipation versus consummation,^[59,60] for learning cue-reward and action-reward contingencies,^[61] and for determining whether expending effort to obtain rewards is worthwhile.^[62] These functions depend heavily on dopamine circuits extending from the ventral tegmental area (VTA), through the striatum (including the nucleus accumbens [NAcc]), and into frontal regions such as the medial orbitofrontal cortex (mOFC), VMPFC, and ACC. Critically, depression is associated with dysfunction in this circuitry,^[63] which is distinct from opioid and endocannabinoid pathways more reliably linked to the experience of pleasure.^[64,65] Indeed, while “anhedonia” suggests reduced pleasure upon reward consummation, accruing evidence instead relates anhedonic depression to blunted anticipatory pleasure ^[66–68], overly conservative calculation of cost/benefit ratios ^[69–71], and deficits in reinforcement learning.^[72,73] This evidence will be succinctly reviewed below.

Behavior

Depression saps motivation to work harder for desirable rewards. One study found a positive relationship between how much a cartoon was enjoyed and how much effort was expended to obtain it in healthy adults, but no such relationship was seen in depressed participants.^[66] Moreover, anticipatory anhedonia was negatively correlated with effort expenditure in this depressed sample, suggesting that failure to anticipate pleasure sapped motivation to work harder. Similarly, when healthy individuals were given a choice between completing an easy task for a small reward or tackling a harder task to earn a larger reward (with only a 50% chance of reward delivery in both cases), increased trait anhedonia predicted fewer choices of the hard task.^[70] A second study using the same task showed that depressed adults made fewer high-effort/high-reward choices than controls, and the number of such choices was negatively correlated with the length of the current major depressive episode.^[71] Finally, another study found that the prospect of increased monetary rewards elicited extra effort on a handgrip task in healthy volunteers, but not depressed adults.^[74] Intriguingly, the depressed group rated themselves as exerting greater effort when more money was at stake even though this was objectively incorrect, suggesting that perceived and actual effort were decoupled. Overall, depressed individuals are unlikely to mobilize extra effort to obtain desirable outcomes, which may reflect anticipatory anhedonia^[66,67], overly conservative cost/benefit calculations^[71], or failure of biological mechanisms that translate incentive motivation into action.^[74]

Anhedonic individuals also have difficulty modifying their behavior as a function of positive reinforcement, suggesting a deficit in reward learning. Our group has developed a probabilistic reward task that uses a differential reinforcement schedule to probe this capacity.^[75] Briefly, participants make a difficult perceptual categorization over the course of several trials, and the probability of reward delivery is three times higher following one response versus the other. In healthy volunteers, this manipulation reliably induces a bias toward the “rich” (more frequently rewarded) response and away from the “lean” response.^[75]

Non-clinical participants with elevated depressive symptoms^[75] and adults with MDD ^[72,76] develop weak response biases, suggesting deficits in reward learning. A trial-by-trial analysis showed that in MDD, the blunted response bias reflected failure to sustain adaptive behavior: depressed individuals stopped choosing the rich response too hastily following non-rewarded trials.^[72] Thus, the depressed participants failed to integrate reinforcement history into their decision-making, and were unable to maintain a response pattern that maximized reward delivery. Importantly, failure to develop a response bias at study entry predicted failure to respond to eight weeks of treatment for depression.^[76]

It is valuable to contrast the effects of depression in these tasks with findings from the sweet taste test, in which participants rate how much they enjoy increasingly concentrated sucrose solutions. This task is one of the purest measures of consummatory pleasure available, as it makes minimal demands on anticipatory responses or reward learning. Strikingly, depression does not strongly affect results from this test^[77–79], and performance is not linked to treatment response.^[78,79] Thus, while motivation to work for rewards and reward learning are negatively affected by depression, basic hedonic responses appear intact.

Neural Circuits

Psychopharmacological and neuroimaging data confirm a role for dopamine signaling in these and related tasks, with implications for anhedonia. For example, in rodents^[80] and healthy humans^[81], low doses of pramipexole (a D2 agonist) blocked response bias development in our probabilistic reward task (Figure 2A). This effect is thought to reflect reduced phasic dopamine release due to activation of pre-synaptic auto-receptors. Consistent with this proposal, an fMRI study using a different task found that low doses of pramipexole reduced striatal and midbrain responses to monetary rewards.^[82] A separate PET study demonstrated that the probabilistic reward task elicits dopamine release in the dACC, mOFC, and VMPFC.^[83] Because the mOFC and VMPFC code reward value in humans,^[84–86] activation of these regions is unsurprising given reward delivery in the task. By contrast, dACC activation is intriguing, especially because source localization of electroencephalography data linked development of a stronger response bias to a neural generator in the dACC.^[87] As already noted, depressed participants performed poorly on this task because they abandoned the more frequently rewarded response too quickly. Work in non-human animals implicates the dACC in the integration of reinforcement history into decision-making^[88], and it may be this psychological process that drives dACC activation (and the link between putative dACC signaling and response bias) in these studies. If so, weaker dACC activation during task performance in depressed versus healthy adults should be detectable via neuroimaging.

Figure 2.

Neural systems implicated in reward processing and anhedonia. (A) Administration of the D2 agonist pramipexole blunts the development of a response bias in the probabilistic reward task, consistent with reduced phasic dopaminergic bursting due to activation of pre-synapatic auto-receptors. Image reprinted from Psychopharmacology 196, Pizzagalli DA, Evins AE, Schetter EC, Frank MJ, Pajtas PE, Santesso DL, Culhane M. “Single dose of a dopamine agonist impairs reinforcement learning in humans: Behavioral evidence from a laboratory-based measure of reward responsiveness”, pages 221–232, copyright 2008, with kind permission from Springer Science and Business Media. (B) Blunted reward learning signals, uncovered with a temporal difference model, during Pavlovian conditioning in medicated depressed adults versus healthy controls. Image reprinted from Kumar P, Waiter G, Ahearn T, Milders M, Reid I, Steele JD. “Abnormal temporal difference reward-learning signals in major depression.” Brain 2008;131:2084–2093, by permission of Oxford University Press. (C) A stronger response to monetary gains in the bilateral dorsal caudate was seen in healthy controls versus unmedicated adults with MDD. Image reprinted from Pizzagalli DA, Holmes AJ, Dillon DG, Goetz E, Birk JL, Bogdan R, Dougherty DD, Iosifescu DV, Rauch SL, Fava M. “Reduced caudate and nucleus accumbens response to rewards in unmedicated individuals with Major Depressive Disorder.” American Journal of Psychiatry 2009;166:702–710. Reprinted with permission from The American Journal of Psychiatry, (Copyright ©2009). American Psychiatric Association.

There is also evidence linking poor reward learning in depression to altered responses in the striatum and ventral tegmental area/substantia nigra (VTA/SN), which contains dopaminergic cell bodies. A combined computational-modeling/fMRI study uncovered weaker reward learning signals in the ventral striatum and dACC of medicated, depressed adults versus healthy volunteers^[73] (Figure 2B). More recently, unmedicated depression was associated with poor reversal learning following unexpected reward delivery, and this was associated with weak fMRI signals in the ventral striatum.^[89] Furthermore, depression had a strong, negative effect on dorsal caudate (Figure 2C) reward responses in the monetary incentive delay task.^[90] The dorsal caudate supports feedback-driven contingency learning^[91], and its activation has been found to normalize with treatment for depression.^[92] Moreover, caudate volume was negatively correlated with anhedonia in clinical^[90] and non-clinical^[93] samples. Therefore, we speculate that depressed adults may have greater difficulty learning action-reward and stimulus-reward contingencies than their healthy peers.

Along these lines, we found an explicit memory advantage for rewarded versus non-rewarded stimuli in healthy individuals, but this effect was not observed in unmedicated, depressed adults.^[94] This group difference was reflected in brain activation during encoding: compared to controls, depressed adults showed weaker reward responses in the VTA/SN. Furthermore, VTA/SN encoding activation predicted memory accuracy in controls, but not depressed participants. Thus, negative effects of depression on dopamine circuitry implicated in reward-driven learning and memory is a promising target for further investigation.

Molecules/Genes

The DAT1 and COMT genes have received significant attention in the context of reward processing. The dopamine transporter (DAT) removes dopamine from synapses, primarily in the striatum, while COMT is an enzyme that degrades dopamine, reducing its synaptic concentration mainly in the PFC. The 9R DAT1 variant is more weakly expressed than the 10R variant, leading to increased striatal dopamine.^{[but see 95]} Meanwhile, the COMT met allele renders the enzyme less stable than the val allele, leaving more dopamine in the PFC. Consequently, several studies have asked whether genotypic variation in DAT1 and COMT affects reward processing.

The answer is “yes”, but a firm understanding of relevant mechanisms remains elusive. On one hand, response bias magnitude in the probabilistic reward task and willingness to expend effort for rewards were greater in COMT met/met versus val/val homozygotes, consistent with a beneficial role for higher PFC dopamine concentrations.^[96] Moreover, compared to DAT1 10R homozygotes, DAT1 9R carriers were quicker to pull a joystick to approach happy faces and push it away to avoid angry faces,^[97] suggesting that higher striatal dopamine concentrations conferred increased sensitivity to rewarding (happy) and non-rewarding (angry) faces. Similarly, the DAT1 9R and COMT met alleles were linked to stronger activation of striatal and PFC regions during reward anticipation and consummation, respectively.^[98] These findings link boosted reward processing with elevated dopamine concentrations.

However, this literature includes many counterintuitive findings. For example, one fMRI study reported a complex gene-gene interaction with regard to ventral striatum reward responses, with weak activation seen in COMT met/met homozygotes with a DAT1 10R allele, as well as COMT val/val homozygotes with a DAT1 9R allele.^[99] Another study found no effect of COMT on reward responses, but possession of the met allele was associated with ventral striatum and temporal pole activation during loss anticipation.^[100] In yet another investigation, COMT val/val homozygotes learned changing stimulus-reward contingencies faster than met/met homozygotes, while also showing stronger reward prediction error signals in the ventral striatum.^[101] Finally, a positive correlation between ventral striatum reward responses and self-reported reward sensitivity emerged in DAT1 10R homozygotes, but not 9R carriers.^[95] These findings are difficult to reconcile with results presented in the preceding paragraph.

The complexity of this literature reflects the relatively small number of studies, small sample sizes, and variations in experimental design. A more fundamental point is that individual differences in reward processing must reflect the influence of myriad genes. For example, in a recent study, about 11% of the inter-individual variance in ventral striatal reward responses could be explained via the additive effect of five genes affecting dopamine signaling, but no single genotype predicted variance when considered alone.^[102]

Summary

Depression is associated with anticipatory anhedonia, unwillingness to work harder for greater rewards, and impaired reinforcement learning—at least when rewards are the reinforcement. These deficits appear to reflect dysfunction in dopamine networks extending from the VTA/SN and into the striatum, as well as in dACC circuitry implicated in cost/benefit analysis and the integration of reinforcement history into decision-making. Genotypic variation that influences dopamine concentration in the PFC (COMT) and striatum (DAT) contributes to individual differences in the responsivity of these circuits, but a clearer understanding of these mechanisms is needed. Although much remains to be done, this evidence supports the decision to focus on several aspects of reward processing (e.g., approach motivation, reward learning) in the RDoC initiative.

III. The Impact of Threats and Stress on Reward Processing in Humans

The first two sections examined threat and reward responses separately, but threats and stress can negatively affect several facets of reward processing. For instance, military training and final examinations blunted responses to amusing films and a wide range of pleasant activities in students.^[103] More recently, healthy adults faced with acute stressors, either in the laboratory^[104] or in natural settings,^[105] showed blunted response bias formation in the probabilistic reward task (Figure 3A), and individuals who perceived their lives as highly stressful developed a weaker response bias than those who did not.^[106] Thus, stress impairs modulation of behavior based on reinforcement contingencies, and recent evidence suggests that these effects may be specific to reward.^[107,108] Finally, an fMRI study demonstrated that acute stress reduces neural responses to rewards (but not punishments) in the human striatum,^[109] highlighting negative effects of stress on reward circuitry.^{[see also 110]}

Figure 3.

Negative effects of stress on reward processing. (A) Acute stress blunts the formation of a response bias in the probabilistic reward task. Image reprinted from Biological Psychiatry 60, Bogdan R, Pizzagalli DA. “Acute stress reduces reward responsiveness: implications for depression”, pages 1147–1154, copyright 2006, with permission from Elsevier and the Society of Biological Psychiatry. (B) Compared to healthy controls, young adults exposed to childhood maltreatment display reduced responses to reward cues in the left globus pallidus. This effect was specific to reward cues, as no group difference was observed in response to loss or no-incentive cues. Image reprinted from Biological Psychiatry 66, Dillon DG, Holmes AJ, Birk JL, Brooks N, Lyons-Ruth K, Pizzagalli DA. “Childhood adversity is associated with left basal ganglia dysfunction during reward anticipation in adulthood”, pages 206–213, copyright 2009, with permission from Elsevier and the Society of Biological Psychiatry. (C) The combination of amygdala hyper-activity during risk anticipation (x-axis) plus NAcc hypo-activation in response to reward delivery (y-axis) characterizes vulnerable individuals, who developed clinically-relevant symptoms of PTSD and/or depression following stress exposure. Note that although amygdala hyper-activity following stress exposure is plotted, the same finding was obtained with amygdala hyper-activity measured prior to stress exposure. Image reprinted from Admon R, Lubin G, Rosenblatt JD, Stern O, Kahn I, Assaf M, Hendler T. “Imbalanced neural responsivity to risk and reward indicates stress vulnerability in humans.” Cerebral Cortex 2013;23:28–35, by permission of Oxford University Press.

Data from non-human animals indicate that reward processing should be most negatively affected by chronic stress,^[111] particularly if it occurs during sensitive developmental periods.^[112] Along these lines, an fMRI study reported reduced anticipatory pleasure in adults exposed to childhood maltreatment.^[113] This was manifested by weak anticipatory responses to reward cues—but not loss or no-incentive cues—in the left putamen and left globus pallidus (Figure 3B). The localization of these findings is noteworthy because globus pallidus lesions result in profound anhedonia and apathy,^[114,115] and damage to left hemisphere basal ganglia structures, especially the pallidum and caudate, is highly predictive of post-stroke depression.^[116] Childhood sexual abuse (CSA) also impaired performance on a reinforcement learning task in adulthood, with the data suggesting a link between CSA and failure to apply reward-based learning in a novel context.^[108] These studies require replication, but they converge with non-human animal data linking chronic stress to dysfunction in brain reward networks.^[111]

Anxious Depression

The findings just described may have particular clinical value, because they suggest a pathway to a burdensome condition: anxious depression, or “cothymia”.^[117] Cothymia is common and places a heavy load on individuals, providers, and society, as it is treatment resistant^[118] and results in extensive healthcare use.^[119] The etiology of cothymia is poorly understood, and neuroscience research is in early stages.^[120] However, results from an elegant prospective study^[121] provide clues for future work, although the study did not directly examine cothymia.

Briefly, healthy members of an elite corps of combat paramedics completed an fMRI study when conscripted and again 18 months later, following exposure to combat-related stressors. The fMRI sessions featured a competitive game that required risky moves and delivered unexpected monetary rewards. The key finding was that two neural markers predicted elevated symptoms of PTSD and depression following stress exposure: heightened amygdala activation during risk anticipation, plus reduced NAcc responses to unpredicted rewards (Figure 3C). Critically, amygdala hyperactivity in stress-sensitive individuals was evident before combat exposure, suggesting it may be a diathesis for stress-induced psychopathology. However, exaggerated amygdala responses alone did not yield a clinical phenotype. Only when combined with NAcc hyporeactivity to rewards—a phenomenon that emerged after stress exposure—was the link to elevated anxiety and depression reliable. This noteworthy study suggests that pre-existing hyperactivity in threat circuitry, combined with stress-induced disruption of reward networks, may be critical for the onset of psychopathology. Although the link to cothymia is speculative, particularly since it does not typically involve such intense stress exposure, this work provides clear neuroscientific hypotheses to pursue in more targeted studies.^[122]

IV. Neural Mechanisms of Stress-Induced Anhedonia

Stress-induced anhedonia suggests interactions between neural systems that mediate threat and reward responses. What is the nature of such interactions, and where do they take place? Initial answers are emerging from diverse sources, including electrophysiological work in non-human primates, investigations of neural metabolism in rodents and humans, and application of deep brain stimulation as a treatment for depression. The most striking data come from a breakthrough new technology: optogenetics, which permits precise manipulation of neurons classified by the neurotransmitters they release. Evidence from these methods is converging on two key findings. First, dopaminergic neurons in the VTA receive inhibitory GABAergic projections from a small sector of the VTA called the rostromedial tegmental nucleus,^[123] and these VTA GABA neurons are excited by the habenula.^[124] Second, the habenula and VTA GABA neurons respond strongly to aversive stimuli. Because of the fundamental role of VTA dopamine neurons in reward processing, these two points suggest a new hypothesis: stress increases metabolism in the habenula, which drives activation of VTA GABA neurons and results in inhibition of VTA dopamine neurons, leading to anhedonic behavior. If this mechanism remained active for a sustained period, it might elicit the onset of anhedonic depression. The remainder of the manuscript reviews new evidence supporting this proposal.

Anatomy, function, and connectivity of the habenula

The habenula is a small, bilateral structure occupying about 30–45 mm³ in each hemisphere in humans (Figure 4A and Figure 4B). It is located at the dorsomedial extent of the thalamus, anterior to the pineal gland, and is bounded medially by the third ventricle.^[124–126] Seminal electrophysiology studies in non-human primates revealed that the lateral habenula’s response profile is the opposite of VTA dopamine neurons: it is excited by punishments, punishment-predicting cues, and reward omission, and inhibited by unexpected rewards and reward-predicting cues.^[127,128] Although the reliability of habenula fMRI signals is unclear due to its small size, there is evidence that this functional profile is conserved in humans.^[129,130] Critically, the habenula is densely connected, receiving inputs primarily from the basal ganglia, hypothalamus, and limbic regions, and sending efferents to the VTA, the raphé nucleus (where serotonergic neurons originate) and the PAG, among other regions.^{[124,130–132]} Although even this thumbnail sketch is complex, the implication is clear: the habenula receives input from regions related to motivation and action selection, and can integrate and broadcast that information via its connections to ascending dopaminergic and serotonergic projections.

Figure 4.

Activation of the habenula and VTA GABA neurons inhibits VTA dopamine neurons. (A) Red box highlights the habenula and neighboring structures in a coronal MRI image. (B) Close-up of the habenula. Images A and B reprinted from Neuroimage, 64, Lawson RP, Drevets WC, Roiser JP. “Defining the habenula in human neuroimaging studies”, pages 722–727, copyright 2013, with permission from Elsevier. (C) Optogenetic stimulation of VTA GABA neurons. The blue bar denotes the stimulation period, the trace depicts the normalized firing rate of 8 neurons (mean: heavy black trace; SD: shaded gray area). (D) Optogenetic inhibition of VTA dopamine neurons during stimulation of VTA GABA neurons. The blue bar denotes the stimulation period, the trace depicts the normalized firing rate of 12 dopamine neurons (mean: heavy gray trace; SD: light gray trace). Images C and D reprinted from Neuron, 73, Tan KR, Yvon C, Turiault M, Mirzabekov JJ, Doehner J, Labouèbe, Deisseroth K, Tye KM, Lüscher C. “GABA neurons of the VTA drive conditioned place aversion”, pages 1173–1183, copyright 2012, with permission from Elsevier.

The most important detail in the current context is that VTA GABA neurons receive excitatory input from the lateral habenula and are positioned to inhibit VTA dopamine neurons.^[123,132] Thus, when the lateral habenula responds to aversive stimuli and excites VTA GABA neurons, inhibition of midbrain dopaminergic neurons is expected. The strongest evidence for this prediction comes from optogenetics, which involves tagging specific cell types (e.g., GABA or dopamine neurons) with molecules that render them sensitive to particular wavelengths of light.^[133] Depending on the preparation, application of that wavelength can selectively excite or inhibit the tagged neurons but not their neighbors, conferring a degree of precision unattainable with conventional methods.

When this approach was used to excite VTA GABA neurons (Figure 4C), a corresponding reduction in the activation of VTA dopamine neurons was observed (Figure 4D), confirming predictions based on anatomy.^[134] Moreover, stimulation of VTA GABA neurons rapidly induced conditioned place aversion, with rats avoiding the chamber in which GABA stimulation had been delivered. This result demonstrates that, in addition to inhibiting VTA dopamine neurons, excitation of VTA GABA cells is aversive. Finally, the same study showed that footshock also induces excitation of VTA GABA cells and inhibition of VTA dopamine cells, lending ecological validity to the optogenetic results.

Confirming a role for the habenula in depression

The work just reviewed raises the possibility that the habenula and VTA GABA neurons may play a role in depressive illness, and several findings support that proposition. One study compared baseline metabolic activity in rats bred to show learned helplessness or resilience in the face of acute stress.^[135] Compared to resilient rats, congenitally helpless rats showed a 64–71% elevation in habenula metabolism, along with a 28% decrease in VTA metabolism (unselected, “normal” rats showed an intermediate metabolic profile). These measures were obtained without stress exposure, suggesting that elevated habenula metabolism and reduced VTA metabolism may be a diathesis for learned helplessness.

Conceptually related work implicates the habenula in human depression. In an early PET study,^[136] acute tryptophan depletion induced depressive relapse in remitted patients, and the increase in depressive symptoms was positively correlated with increased habenula activation. Similarly, a placebo-controlled fMRI study reported increased habenula responses to emotionally negative words following acute tryptophan depletion in a sample of unmedicated, remitted depressed participants.^[137] Based on these studies and the encouraging findings in non-human animals, the habenula has been proposed as a target for deep brain stimulation (DBS) in depression.^[138] Importantly, DBS would be used to inhibit the habenula. Application of this approach in a single case of treatment-resistant depression yielded promising results,^[139,140] which should encourage follow-up work in larger samples.

Of course, the majority of depressed patients are not candidates for DBS, thus these findings might appear to have limited relevance for most treatment decisions. However, baseline serotonin transporter concentration in the habenula (relative to the median raphé nucleus) predicted response to selective serotonin reuptake inhibitors (SSRIs),^[141] indicating that assessment of habenula physiology could inform the selection of front-line antidepressants. This development lies in the future, but it reflects accumulating evidence of a central role for the habenula in human depression.

Optogenetics and anhedonia

Investigating connectivity between the habenula and serotonergic neurons in the raphé nucleus will clearly prove important for understanding depressive illness. Moreover, a study of human pain^[130] revealed functional and structural connections between the habenula and the PAG. As noted earlier, the PAG is reliably elicited by imminent threats, and increased PAG volume has been linked to panic disorder. Therefore, the habenula may contribute to aversive signaling in a manner that extends beyond depression, and this may primarily reflect its ability to modulate serotonergic firing.

However, any relationship between habenula function and anhedonia should be mediated through its effects—via VTA GABA cells—on VTA dopamine neurons. Along these lines, an important optogentic study^[142] showed that anhedonic symptoms of depression can be influenced by modulating activity in VTA dopamine neurons. Two experiments from this elegant study are particularly noteworthy. In the first, VTA dopamine neurons were selectively inhibited as rats underwent the forced swim test and a measure of sucrose preference. Strikingly, optogenetic inhibition of VTA dopamine neurons rapidly induced cessation of struggling in the swim test (Figure 5A) and dramatically reduced sucrose preference (Figure 5B). These classic markers of depression-like behavior and anhedonia disappeared shortly after optogenetic stimulation was stopped. Thus, inhibition of VTA dopamine neurons induced a pro-depressive phenotype.

Figure 5.

Inhibition of VTA dopamine neurons induces a pro-depressive phenotype, stimulation of VTA dopamine neurons elicits an anti-depressive phenotype. Optogenetic inhibition of VTA dopamine neurons reduces time spent struggling in the forced swim test (A) and sucrose preference (B). Orange traces show data from mice treated with a light-sensitive molecule that hyperpolarizes cell membranes upon illumination, gray traces show data from controls, and the yellow panel delimits the period of illumination. Exposure to chronic mild stress (CMS) reduces time spent struggling in the forced swim test (C) and sucrose preference (D), but optogenetic stimulation of VTA dopamine neurons rescues performance. Light blue traces show data from mice exposed to CMS and treated with a light-sensitive molecule that depolarizes cell membranes upon illumination, gray traces show data from mice exposed to CMS but not treated with the light-sensitive molecule, and purple and black traces show data from controls not exposed to CMS (purple: carrying the light-sensitive molecule; black: no light sensitive molecule). Blue panel delimits the period of illumination. Images A, B, C, and D reprinted by permission from Macmillan Publishers Ltd: Nature, 493, Tye KM, Mirzabekov JJ, Warden MR, Ferenczi EA, Tsai H-C, Finkelstein J, Kim S-Y, Adhikari A, Thompson KR, Andalman AS, Gunaydin LA, Witten IB, Deisseroth K. “Dopamine neurons modulate neural encoding and expression of depression-related behaviour”, pages 537–541, copyright 2013.

In the second experiment, mice were first exposed to the chronic mild stress (CMS) paradigm. This involved twice-daily exposure to a battery of minor stressors over a period of 8–12 weeks, which is known to induce anhedonic behavior.^[111] Indeed, compared to controls, mice exposed to CMS displayed a weaker sucrose preference and ceased struggling earlier in the forced swim test. Strikingly, however, these pro-depressive effects were rescued within minutes by optogenetic stimulation of dopamine VTA neurons (Figures 5C and 5D), which normalized the behavior of CMS-exposed mice to the level demonstrated by non-stressed animals.

This optogenetic investigation suggests two complementary points: inhibiting VTA dopamine neurons induces anhedonic, pro-depressive behavior, while exciting VTA dopamine re-instates normative reward responses and anti-depressant behavior in mice exposed to CMS. Together with the aforementioned demonstration that excitation of VTA GABA cells inhibits VTA dopamine neurons, and the data linking stress to blunted reward processing in humans, this work highlights a neural mechanism that transduces stress and threat responses into anhedonic behavior. Whether the mechanism differs substantially in humans is a crucial question, but it is encouraging to note that both psychotherapy^[92] and pharmacotherapy^[143] are capable of rescuing striatal function in depressed adults. Thus, remediating stress-induced dysfunction in reward pathways may be a viable goal for treatment interventions.

Cautions and complexities

For any translational effort to be successful, additional complexities must be acknowledged. Most importantly, the relationship between activation of VTA dopamine cells and depressive behavior appears to depend critically on the stress protocol. This was made clear when the same group used optogenetics to modulate VTA dopamine neurons in mice exposed to social defeat stress, rather than CMS.^[144] In the social defeat paradigm, a test mouse is introduced into the cage of a larger, “resident” mouse, who typically attacks. When done acutely, this manipulation renders mice susceptible to further stressors but does not typically induce anhedonia.^[144] However, when this “sub-threshold” manipulation was accompanied by phasic bursting of VTA dopamine neurons (elicited via optogenetics), a robust increase in depressive symptoms was observed: social interactions decreased, as did sucrose preference. When the social defeat paradigm is administered chronically (e.g., over 10 days), anhedonia and depressive behaviors are elicited in about 50–60% of mice, with the remainder displaying resilience.^[145] Strikingly, however, this study also showed that if chronic social defeat is paired with VTA dopaminergic bursts, resilient mice can be converted into susceptible mice, as measured by decreased social interaction and sucrose preference. By contrast, inhibition of dopamine circuitry extending from the VTA to the NAcc reduced depressive behaviors following chronic social defeat. Broadly speaking, these results are the opposite of those obtained with CMS.^[142]

It is not easy to reconcile these two sets of findings, but the nature of the stressor may prove critical.^[146] In particular, while chronic stress reduces dopamine bursting, acute stressors elicit it^[147], and this may alter the impact of additional bursting induced by optogenetics. Alternatively, more prosaic differences between the experiments (e.g., single vs. group-housing of animals) may prove relevant.^[146] Regardless, additional work is needed.

Indeed, we have glossed over other complexities. For example, although most VTA dopamine neurons show the “classic” responses to unexpected rewards and reward-predicting cues, others respond to rewarding and aversive stimuli and carry signals related to salience rather than valence.^[148,149] Determining whether these neurons are involved in depression is a goal for the future. Finally, we have emphasized negative effects of stress and threat on dopaminergic firing, but dopaminergic firing can also delimit stress and threat responses. For example, a study in knockout mice showed that stimulus control over fear responses is lost if dopaminergic responses to aversive stimuli are inhibited.^[150] Given evidence of poor discriminative learning and generalized fear responses in anxiety disorders, this work suggests a possibly underappreciated role for dopamine in human anxiety.

V. Conclusion

This review integrated research on threat and reward processing from several levels of analysis, consistent with the RDoC approach, and emphasized novel findings concerning the intersection of stress, threat, and reward processing. We close with three final points. First, our focus on threat and reward is not meant to suggest that other RDoC domains are not equally important for understanding anxiety and depression. Second, by emphasizing behavioral, molecular genetic, and neural factors in mental illness, we do not mean to understate the role of the environment. Finally, for RDoC research to inform clinical practice, additional emphasis must be placed on replication studies.^[52] Academic institutions and funding bodies favor novelty, but clinical practice demands reliability. Thus, replication is essential, in order to identify robust findings with maximal clinical value.