Reward Networks in the Brain as Captured by Connectivity Measures

Abstract

An assortment of human behaviors is thought to be driven by rewards including reinforcement learning, novelty processing, learning, decision making, economic choice, incentive motivation, and addiction. In each case the ventral tegmental area/ventral striatum (nucleus accumbens) (VTA–VS) system has been implicated as a key structure by functional imaging studies, mostly on the basis of standard, univariate analyses. Here we propose that standard functional magnetic resonance imaging analysis needs to be complemented by methods that take into account the differential connectivity of the VTA–VS system in the different behavioral contexts in order to describe reward based processes more appropriately. We first consider the wider network for reward processing as it emerged from animal experimentation. Subsequently, an example for a method to assess functional connectivity is given. Finally, we illustrate the usefulness of such analyses by examples regarding reward valuation, reward expectation and the role of reward in addiction.

Introduction

The investigation of the behavioral and neural consequences of rewarding (reinforcing) and punishing events has a long standing history, dating back to the early investigations of operant conditioning. Recent neurobiological research has suggested that important aspects of reward processing are coded by dopaminergic neurons arising from the ventral tegmental area (VTA) and projecting to the ventral striatum (VS) via the mesolimbic pathway. Interestingly, the VTA–VS dopamine system has been found to be of eminent importance in a variety of motivated behaviors and cognition. For example, it has been implicated in reinforcement learning (Schultz, 1998), action monitoring (Holroyd and Coles, 2002; Kramer et al., 2007), novelty processing and learning (Schott et al., 2004b), decision making and economic choice (McClure et al., 2004), incentive motivation (Berridge and Kringelbach, 2008), and addiction (Heinz et al., 2004; Reuter et al., 2005). In turn, all of these processes have been linked to reward processing. This is surprising, because the VTA is a comparatively small assembly of cells (with about 400000 cells in the adult human). The question thus arises as to how the VTA–VS system can modulate the wide spectrum of behaviors mentioned above. One possibility which we will elaborate on in this article is that the VTA–VS system is part of a wider network of brain structures. Depending on the specific context, activity in the VTA–VS system may interact with different other subcortical and cortical brain areas which could be the basis of the flexibility of this system.

The identification of a particular brain region with a specific cognitive function has been a central topic in neuroscience. Indeed, the major goal of functional magnetic resonance imaging (fMRI) analyses is to capture the blood oxygenation level-dependent (BOLD) signal associated with a particular task-related neural activity. In many fMRI experiments, multiple areas are found to be coactivated during a given task. However, standard univariate MRI analysis is not able to capture the task-related dynamics within a network of brain areas. A more complete understanding of the brain processes associated to a specific process requires both regionally specific activations and regionally specific interactions.

In the following, we will first review the key structures that are involved in reward-related behavior, including its relation to the learning and motivational circuits. We will then give an example for a method that can be used to assess interregional connectivity in conjunction with fMRI. The reward-related networks as evident for reward valuation, reward anticipation and addiction will be briefly discussed.

Key Structures Mediating Reward-Related Behavior

The desire to maximize rewards and to minimize negative possible outcomes is an important drive of human behavior. Because of this, humans are motivated to identify and seek possible cues in the environment which might predict the possible appearance of rewards or negative outcomes, as well as behaviors which could cause the appearance of these outcomes. The association of an event with a reward or a punishment therefore constitutes a powerful learning signal. In addition, we use information from the feedback signals elicited by our actions to influence our future decisions. In ambiguous contexts and situations in which different outcomes are probable or when feedback information is not available, humans might need to make decisions which can be considered risky, erratic or impulsive, sometimes irrationally pursuing short-term pleasures without considering that these actions could lead to negative after-effects in the future. Recent work in experimental economics (Glimcher et al., 2005) and decision making science (Schall, 2005) suggests that there are large interindividual differences with regard to the way we deal with rewards and punishments of different magnitude in certain situations. Interestingly, the cognitive processes required for successful adaptation in these situations might require the elicitation of affective responses (emotional valuation), the ability to associate neutral events to the appearance of an emotional-charged outcome (learning) and the ability to store this information in order to make predictions (memory). This intersection between affective processes, learning and memory is a core aspect of reward processing, motivated behavior and decision making in humans.

A primary challenge in affective neuroscience is to understand to which degree these processes are subserved by specific brain regions or by common, partially overlapping networks. Indeed the ultimate aim would be to describe the specific role of each brain region and how the specific information computed in each brain region is assembled by larger brain midbrain-limbic-(sub)cortical networks in a process-specific way. A main problem encountered by the standard functional imaging approach to reward processing is that a large number of activations are usually seen. Reward processing thus consistently increases the BOLD response in a common set of regions comprising the VS (the nucleus accumbens, NAcc), the amygdala, the prefrontal cortex (orbitofrontal cortex, OFC), and the insula (Delgado et al., 2000, 2003; Breiter et al., 2001; Knutson et al., 2001, 2003b; McClure et al., 2004; Yacubian et al., 2006; Tom et al., 2007). Several studies have also identified activations in the midbrain regions (see for a review Duzel et al., 2009) as well as the ventromedial prefrontal cortex (vmPFC) or anterior cingulate cortex (ACC), although less consistently (Knutson et al., 2003b; Sanfey et al., 2003; Ullsperger and von Cramon, 2003).

The NAcc and VTA are placed prominently within a network that is not only implicated in the immediate processing of rewards but also in learning and motivation (Figure 1, see also Figure 1 in Münte et al., 2008).

Figure 1.

Reward processing networks also involved in learning, memory and addition. Green boxes highlight the hippocampal-VTA learning-memory circuit described by Lisman and Grace (2005). The motivational system has been adapted from Kelley (2004) (yellow boxes). Notice that the direct and indirect projections from the hypothalamus to the neocortex–limbic structures through the dorsal thalamus is omitted from the figure. vmPFC, ventromedial prefrontal cortex; OFC, orbitofrontal cortex; PPTg, pedunculopontine tegmentum; LTP, long-term potentiation; v, ventral.

The learning hippocampal-VTA (HP-VTA) loop (Figure 1, green boxes) has been adapted from Lisman and Grace (2005) who have proposed that hippocampal novelty signals might be conveyed to the midbrain (SN/VTA) through the NAcc and the ventral pallidum. The role of the ventral pallidum as an essential region involved in liking sensations has also been highlighted (Berridge and Kringelbach, 2008). This loop is important for encoding predictions based on stimulus-novelty. Novelty detected by the hippocampus might be sent through the subiculum, NAcc and ventral pallidum to the dopaminergic midbrain regions. Phasic activity in these midbrain neurons in primates have been observed to change according to the delivery of and expectation for salient and rewarding events (Schultz, 1998). Specifically, increases of DA cell firing have been associated to positive outcomes, whereas choices that did not lead to a reward evoked dips in the firing rate below baseline (Schultz, 2002). This phasic firing might result in release of dopamine in the hippocampus where it might enhance long-term potentiation, and as a consequence, memory storage and learning. Notice also, that midbrain dopaminergic system projects to and thus modulates other striatal-orbitofrontal and prefrontal regions involved in reward processing through the mesocortical and mesolimbic pathways (Apicella et al., 1991; Hikosaka and Watanabe, 2000; Wise, 2002). The mesocortical pathway projects primarily to the vmPFC, ACC and entorhinal cortex. The mesolimbic pathway directly innervates the NAcc, septum, olfactory tubercle, amygdala and piriform cortex. As a confirmation of the importance of this HP-VTA loop in learning and memory, the activation of the substantia nigra/VTA and the hippocampus has been recently associated with novelty processing and facilitation of memory formation (Schott et al., 2004a, 2006; Wittmann et al., 2008). Similarly, in a reward-motivated memory formation task (Adcock et al., 2006), high-reward cues preceding remembered but not forgotten scenes activated VTA, the NAcc and the hippocampus.

The second “motivational” circuit (Kelley et al., 2005) allows the organism to seek specific stimuli needed for survival by producing spontaneous locomotor behavior and exploration, ingestive, defensive and reproductive behaviors. These systems have been recently integrated in what is termed the “behavioral control columns” (Swanson, 2000), which are defined as a set of highly interconnected nuclei in the hypothalamus and its brainstem extensions devoted to the elicitation and control of specific behaviors necessary for survival (see Figure 1, yellow boxes). These motivational systems might be activated by specific environmental (internal or external) stimuli and are amplified and energized by affect or emotion. During evolution, these hard-wired hypothalamic-brainstem circuits have been progressively interconnected with phylogenetically more recent structures such as the PFC, striatum and limbic regions, allowing the implementation of cognitive control and more flexible motivated behavior. Massive direct and indirect afferents from the hippocampus, amygdala, VS and PFC project to the behavioral control columns, allowing the implementation of highly complex cognitive processes. For example, the amygdala, which is considered a key structure in emotional valuation, projects to the lateral hypothalamus and removal of this amygdalo-hypothalamic pathway does not abolish food intake per se but it alters the assessment of the comparative value of the food based on learning (Petrovich et al., 2005). Importantly these hypothalamic structures project to the midbrain dopaminergic neurons which in the case of expectation and consumption of primary and secondary rewards might elicit the activation of the NAcc and the PFC. Importantly, the hypothalamic behavioral control subsystems project massively back to the cerebral cortex via the dorsal thalamus (not shown in Figure 1). These feedforward projections provide higher order cortical centers with access to internal motivational states.

Notice, that in both circuits the NAcc is a key integrative region weighting the different inputs coming from cortical areas (OFC, vmPFC, DLPFC, insula), limbic regions (amygdala, hippocampus; Groenewegen et al., 1999) and midbrain (SN/VTA) and therefore modulating the selection of appropriate responses and goal-directed behavior (Berridge and Robinson, 1998; Goto and Grace, 2005; Kelley et al., 2005). Moreover, the interactions of the medial prefrontal cortex (ACC) and the VS (both receiving DA input from the midbrain) in the adjustment of behavior have been highlighted (Holroyd and Coles, 2002).

Functional Connectivity Measures

Obviously, Figure 1 presents the basic network in which the VTA–VS reward system exerts its influence on different behaviors. The key question is, how the different elements within this network work together in different behavioral contexts. This question might be answered by studying connectivity patterns and indeed cognitive neuroscience has increasingly acknowledged the need for a network approach (Rykhlevskaia et al., 2008). Accordingly, a growing number of neuroimaging studies have shifted the focus from standard univariate to connectivity analyses. Functional connectivity is defined as the statistical association or dependency among two or more neurophysiolgical time-series recorded in spatially remote areas (Friston, 1994; Horwitz, 2003). Initial functional connectivity studies (PET and fMRI) used correlation analysis between a small number of pre-selected regions or between voxels of interest in order to study functional connectivity (Biswal et al., 1995). As fMRI uses an indirect measure of neurophysiological functioning (BOLD signals), inter-regional dependencies can be investigated using correlation of BOLD signals between remote areas. Those regions showing large correlations are considered functionally connected. Correlation between two regions might exist even in the absence of a direct connection, therefore mediated by a third region. Partial correlation measures could be used in this particular case for removing the contributions of pair-wise correlations that might arise due to global or third-party effects (Hampson et al., 2002; Sun et al., 2004; Salvador et al., 2005).

When two regions are active roughly at the same time, then the two BOLD time-series might be highly correlated. In this particular case, immediate instantaneous or zero-order correlation measures between the two time series will capture the relationship between these signals (Hampson et al., 2002). However, when one signal is delayed from the other but showing a similar fluctuation, a time-shifted or lagged cross-correlation analysis is needed in order to capture the possible linear but delayed relationship between these regions. Notice that because of the characteristics of the BOLD response, the correlations are based on low-frequency fluctuations. In a functional connectivity correlational study Cordes et al. (2001) showed that over 90% of their connectivity were due to low-frequency (below 0.1 Hz) fluctuations in a block-design paradigm. One important caveat of simple correlation analysis is that this measure is highly sensitive to the shape of the hemodynamic response function, such as onset-delay, time-to-peak, and width, which are region-specific due to differences in vascular properties across regions (Buckner et al., 1996; Bandettini and Cox, 2000). Because of that, this method is mostly appropriate to block-design analysis in which the shape of the hemodynamic response function shows less variability.

In contrast, coherence-related measures are less prone to the shape of the hemodynamic response function, as they are equivalent to the cross-correlation-related approaches, but using information from the frequency-domain. Cross-coherence measures have been shown to be very useful in investigating functional connectivity across brain regions (Leopold et al., 2003; Sun et al., 2004; Salvador et al., 2005). Because the analysis is performed in the frequency domain, this measure is blind to the possible lags of one region when compared to another one. In this sense, if the frequency content of one series is similar to another one, then the spectral coherence will be large indicating strong connectivity between two regions.

Because the restriction of cross-correlation connectivity analyses to block designs, a new methodological approach has been introduced to characterize functionally interacting regions using event-related fMRI designs (Rissman et al., 2004). This approach is based on the parameter estimates obtained in the context of the general linear model. Within this approach, a series of parameter estimates is extracted from a seed region and correlated with voxels from the whole-brain. Using this method, it is possible to identify specific functionally related brain networks. Similar solutions have been proposed by other authors (e.g., Siegle et al., 2007; Aizenstein et al., 2009).

In recent years, many improvements have been made also in the description and localization of functional patterns (for a review, see Rogers et al., 2007). Some concerned the reduction of the number of regions involved in the correlation analysis. As the number of regions that are of interest increases, the covariance matrix becomes increasingly larger and thereby computations become more complex and more difficult to interpret. Indeed, different statistical multivariate approaches have been used to simplify the model, such as multidimensional scaling, principal component analysis, independent component analysis, and principal least squares, among others. These methods are very attractive in the sense that they do not require any prior hypothesis about the connectivity links of interest.

When studying functional connectivity it is also worth to consider the possible presence of spontaneous correlations between different brain regions. For example, Biswal et al. (1995) showed consistent correlations between different parts of the brain (bilateral primary motor and supplementary motor regions) during resting states (i.e., when a participant is not performing any particular task; see similar results in Xiong et al., 1999; Cordes et al., 2000; Lowe et al., 2000). In a subsequent study, Hampson et al. (2002) investigated the changes in functional connectivity induced by a task (listening to continuous speech) when compared to a resting condition. Interestingly, higher correlations were observed between Broca's and Wernicke's regions when participants were actively listening to speech. The description of a consistent “default-mode” network in the resting brain in the absence of any stimulus (Raichle and Snyder, 2007) implies that such networks have to be taken into account when evaluating regional BOLD correlations during task conditions (Hampson et al., 2002). It is possible that these coherent spontaneous oscillations might account for a fraction of the trial-to-trial variability in BOLD event-related responses (Fox et al., 2006).

Finally, it is important to remind oneself that the presence of a structural connection makes a functional connection biologically meaningful and more likely to occur. Therefore, analyses of anatomical connectivity open a useful tool for restricting the number of functional connections to be analyzed. Until recently, most connectivity approaches did not take into account the details of anatomical connectivity. However, if two regions are not anatomically connected, a functional connection is biologically implausible. Moreover, it is reasonable to expect that the strength of anatomical connections might modulate the corresponding functional connections. Traditionally, anatomical connectivity maps have been restricted to animal invasive histological experimentation (Beaulieu, 2002) but the advent of diffusion tensor imaging combined with the development of new analysis tools opens up new opportunities. DTI based tractography provides detailed information of the structural connections (Hagmann et al., 2007). To use functionally defined seed points for fiber tracking algorithms appears very promising to investigate the direct relation between brain function and structure (Staempfli et al., 2008). Indeed, two recent studies that combined structural (DTI) and functional connectivity measures have shown a high degree of similarity between both connectivity estimates (Skudlarski et al., 2008; Honey et al., 2009).

The functional connectivity measures discussed thus far are uninformative about the causality or directionality of the influence between the different brain regions, i.e., what it is known as “effective connectivity” (Friston, 1994). Causality is taken into account by another set of methods such as structural equation modeling, dynamic causal modeling and psychophysiological interaction analyses (Friston et al., 1997). These approaches constrain the connectivity analysis to a limited number of regions, based on prior knowledge (model based) about anatomical connections or functional systems (for a review on these methods, see Horwitz et al., 2005).

In sum, connectivity measures may greatly enhance our ability to map brain activations to behavior. However, several important limitations have to be considered. First, the BOLD response is a rather indirect measure of the brain at work. In particular, the question arises as to which aspect of the neural activity of the reward area is reflected in the BOLD response. With regard to the BOLD signal in the NAcc, in a thoughtful review Knutson and Gibbs (2007) have convincingly suggested that it is modulated by dopamine signals arising from the VTA. Dopamine is released in the NAcc and shows a tendency to diffuse over wide areas (Garris et al., 1994) stimulating presynaptic D2-type autoreceptors and D1/D2-type postsynaptic receptors. Animal experiments have shown that the onset of changes in the membrane potential of postsynaptic neurons is around 200 ms and lasts for about 1000 ms after a single neural impulse. Knutson and Gibbs (2007) further argue that the average firing rate of five impulses per second should lead to changes in extracellular dopamine levels on a second-to-second basis. That these changes in extracellular dopamine influence the BOLD signal has been further substantiated by animal experiments which showed that NAcc extracellular dopamine and the BOLD signal had a similar temporal profile and that lesioning of dopaminergic neurons also abolished the NAcc BOLD response (Chen et al., 1997). With regard to humans, it has been demonstrated by many studies that the BOLD signal to rewards or reward cues peaks at about 4-6 s (Knutson et al., 2003a; Camara et al., 2008; Riba et al., 2008). Knutson and Gibbs (2007) suggest a time-line connecting dopamine release and fMRI BOLD response as follows: (a) dopamine is released and activates postsynaptic D1 and D2 receptors 0-2 s after firing; (b) this changes the postsynaptic membrane potential (0-2 s after firing) which (c) requires energy and oxygen from nearby capillaries, which (d) is followed by an increase of the BOLD signal 4-6 s after firing. Thus, we can be reasonably sure that the BOLD response in the NAcc tracks changes in dopamine level over time. Simultaneous recordings of electrophysiological signals and the BOLD response in animals have suggested that in cortical areas the BOLD response is related to local field potentials rather than multi-unit activity (Logothetis et al., 2001; Logothetis and Wandell, 2004). This suggests that connectivity between the NAcc and cortical areas may reflect the dopaminergically modulated influence of the NAcc on these areas.

A second problem is the slowness of the BOLD signal and the fact that data-points are obtained approximately every 1.5-2 s. Therefore, it might well be that fMRI-based connectivity measures underestimate the degree of interregional exchange in the brain. A promising complementary line of research to elucidate the mechanisms that sustain connectivity in the brain is the investigation of neurophysiological oscillations in different brain regions. Synchronous oscillatory activity in distant regions might be a mechanism that sustains functional connectivity. For example, synchronous oscillatory activity within subcortical and cortical networks have been related to learning and decision making (Paz et al., 2006; Pesaran et al., 2008). In a recent study, Popescu et al. (2009) showed learning-related increases in gamma coherence between the basolateral amygdala and the ventral putamen, using local field potentials recorded in cats performing an appetitive learning task. Analysis of electrophysiological activity has also demonstrated that communication between distant brain regions may also be established by phase-locking in different frequency bands. Note, that such synchronization processes might not necessarily be associated with an increase in metabolism and a change in the BOLD signal, and in this sense functional connectivity fMRI methods might be limited to investigate these issues. Animal experimentation is therefore needed to explore the limits of connectivity assessment using fMRI.

Human electrophysiological investigations may provide interesting insights on interregional communication in particular with regard to reward. For example, several studies have already shown oscillatory activity in the theta, beta and gamma bands in humans related to reward processing using non-invasive measurements (Cohen et al., 2007; Marco-Pallares et al., 2008). There is also a small but growing number of studies that have used simultaneous recordings from intra cerebral electrodes in the NAcc and surface electrodes (Münte et al., 2007; Cohen et al., 2009a,b,c; Heinze et al., 2009) which took advantage of the possibility to assess correlations between depth and surface electrodes. Such investigations, while limited in the areas that can be reached with intracerebral electrodes by clinical considerations, may provide crucial timing information. For example, in a recent intracranial study in awake humans, Cohen et al. (2009c) showed increased theta activity in the NAcc in monetary loss feedback trials but not in gain trials in a reversal learning task. In these “loss” trials, participants had to adjust their behavioral strategy in order to gain more money. This study provided compelling evidence about the role of the VS in behavioral adjustment as it is clearly responsive to negative feedback that signals the need of such a readjustment.

Empirical Data: The Brain's Reward System in Different Contexts

A network supporting reward valuation

The valuation of monetary gains and losses activates a similar fronto-subcortical-limbic network, but to a different degree. Specifically, large activations have been reported in the VS, the cingulate cortex, the superior frontal cortex, the inferior parietal lobule, the insular cortex, hippocampal regions, the thalamus, and the caudate nuclei (Delgado et al., 2000, 2003; Gottfried et al., 2003; May et al., 2004; Nieuwenhuis et al., 2005; Dreher, 2007; Marco-Pallares et al., 2007; Tom et al., 2007; Camara et al., 2008). However, it is still controversial to which degree the neural mechanisms underlying reward and punishment valuation recruit different brain regions. For example, Frank et al. (2004) proposed a differential modulation of excitatory and inhibitory pathways in the VS by positive and negative outcomes. Similarly, Wrase et al. (2007) have reported differences in adjustment of motor responses after the delivery of rewards compared to punishments. In this sense, the examination of brain connectivity patterns might help to differentiate the networks engaged in the processing reward and losses.

We recently conducted as study to address this issue. Importantly, functional connectivity results showed a different mesolimbic network than previous classical univariate analyses (Camara et al., 2008). In this study a gambling task required participants to bet on one of two sums of possible money which could be gained or lost (win or loss feedback appeared after the participant's decision). Occasionally, unexpectedly large sums were won or lost, which where five times larger in magnitude than the standard wins and losses but occurred in only 10% of the trials. Functional connectivity analyses showed an extensive network of regions supporting similar responses to reward and punishment valuation including the insular cortex and OFC, the amygdala, the hippocampus and the SN/VTA midbrain regions. Notice, that this network clearly engaged the HP-VTA learning circuit proposed in Figure 1 (Lisman and Grace, 2005) (see also Figure 2A,B). These regions correlated with the activity observed in the VS seed region (NAcc), which was the region which was selected as a seed point in order to perform the functional connectivity analysis. For losses stronger correlations were found between the VS and the medial OFC, indicating a relatively stronger relationship between these structures in the valuation of punishments. Moreover, there was a tendency for a greater involvement of the amygdala in the network elicited by losses (see Figure 2B).

Figure 2.

Nucleus accumbens connectivity during reward valuation. (A) Scheme of the reward valuation network in light of Camara et al. (2008) results (yellow boxes, black arrows) embedded in a wider motivation/learning circuit (gray boxes and arrows). The wider network is slightly modified (omitting unspecific hypothalamic/thalamic projections) from Kelley (2004). (B) Regions functionally connected with the nucleus accumbens (NAcc) after unexpectedly large sums were won or lost are superimposed on a group-averaged structural MRI image in standard stereotactic space (T-score overlays). Gains and losses connectivity patterns are simultaneously depicted: gain (green, P < 0.001), loss (red, P < 0.001) and conjunction gain ∩ loss (yellow, P < 0.001 and P < 0.001).

These results complement a previous connectivity study (Cohen et al., 2008) in which micro-structural properties of white matter tracts were predictive of functional connectivity after reward delivery. Importantly, the projections connecting the amygdala with the hippocampus, the OFC, and the VS not only predicted connectivity derived from fMRI time series but also participants’ behavior following both positive and negative feedback in a reversal learning task (Cohen et al., 2008). One important aspect is that these results highlight the involvement of the VS (NAcc) as a key region in the motivational network developed on the basis of animal research (Kelley et al., 2005) (Figure 2). The different patterns obtained using the classical univariate analysis and connectivity analysis suggest that different information is retrieved using these two methods and stresses the importance of using functional connectivity as a complementary tool (Gazzaley et al., 2004; Rissman et al., 2004; Buchsbaum et al., 2005; Ranganath et al., 2005; Fiebach and Schubotz, 2006). An important aspect which was not analyzed in our previous study is whether other networks could have been identified if a different seed region would have been chosen. For example, the BOLD response in the vmPFC cortex did not correlate with the NAcc activation, which might suggest that a different network could be identified and related to a different functional role.

A network supporting reward expectation

As well as the processing of reward outcomes, the expectation of primary (O'Doherty et al., 2002), monetary (Knutson et al., 2000, 2001), and social rewards (Izuma et al., 2008, 2009; Spreckelmeyer et al., 2009) is supported by similar fronto-subcortical-limbic networks, including the VS (NAcc) and the PFC, including the insular cortex (see Fehr and Camerer, 2007; Knutson and Greer, 2008 for reviews). Among these regions, the NAcc plays a primary role and is more activated for cues signaling potential rewards than cues signaling no reward. This anticipation effect has been linked to dopamine transmission in the NAcc (Knutson and Greer, 2008; Schott et al., 2008) and can be modulated by altering dopamine reuptake (Scheres et al., 2007; Strohle et al., 2008), changing dopamine breakdown (Yacubian et al., 2007), or using dopamine receptor agonists or antagonists (Abler et al., 2007). The PFC is assumed to control impulsive behaviors, being important for emotion regulation during decision making (McClure et al., 2004). It has been reported that individuals who tend to continue previously rewarded behaviors (rather than impulsive behaviors) show stronger structural connectivity between the striatum and the prefrontal cortex (Cohen et al., 2008). The insular cortex, on the other hand, is mainly associated with emotional processes and interacts with the VS during reward delivery (Camara et al., 2008).

In a recent fMRI study using functional connectivity (Ye et al., article submitted) we further showed that these regions interact as a network during reward expectation. Moreover, this network can be distorted by dopamine receptor agonists such as pramipexole¹, which is widely prescribed to treat Parkinson's disease but has been reported to cause pathological gambling as well as other impulse control disorders (Dodd et al., 2005; Weintraub et al., 2006). More specifically, intensive functional connectivity was observed between the NAcc and the PFC during the anticipation of monetary rewards (see Figure 3, placebo condition). This prefrontal–striatal connectivity, however, is reduced by the administration of pramipexole. Instead, the connection between the insular cortex and the VS is enhanced (see Figure 3, pramipexole condition). The weakened connectivity between the VS and the prefrontal cortex may lead to an impaired top-down executive control of impulsive behaviors, while the enhanced connectivity between the VS and the insular cortex may amplify the emotional influences on decision making (see Figure 2, schemes). Indeed, the role of the vmPFC cortex in emotion regulation is well established, projecting directly to the amygdala and most probably providing some inhibitory input (Quirk and Beer, 2006). This shift in connectivity patterns may give rise to an overestimation of potential rewards but to an underestimation of possible risks. The imbalance between the prefrontal–striatal circuitry and the insula–striatal circuitry may explain why pramipexole treated patients tend to develop pathological gambling and other impulse control disorders. The Ye et al. results are consistent with predictions that follow from the tonic-phasic dopamine hypothesis proposed by Grace and colleagues (Grace, 1991; Bilder et al., 2004). This hypothesis assumes that dopamine dynamics in the striatum are driven by the interactions of phasic and tonic dopamine release. Pramipexole may reduce phasic dopamine release by activating dopamine autoreceptors D2/D3 and at the same time change tonic dopamine release by affecting prefrontal–striatum glutamatergic projections. It has been reported that the stimulation of cortical dopamine D2 receptors may directly inhibit the activity of glutamate neurons in the prefrontal cortex and subsequently the activity of dopamine neurons in the NAcc, eventually leading to a decrease in extracellular dopamine level (Beyer and Steketee, 2000; Del Arco and Mora, 2005). To compensate the change in dopamine receptor stimulation, the amplitude of dopamine efflux is increased. The effect of pramipexole on phasic processes may be overridden by the effect of pramipexole on tonic processes, resulting in the increased NAcc activity during reward anticipation. The increased NAcc activity may reflect exaggerated incentive responses to possible rewards, and could be followed by impulsive behaviors and suboptimal choices (Kuhnen and Knutson, 2005).

Figure 3.

Nucleus accumbens connectivity during reward expectation. Regions functionally connected with the NAcc during reward expectation under placebo and pramipexole. Arrows indicate the frontal cortex (blue) and the insular cortex (green). Left hemisphere is on the left side. Color scale refers to T values. Bottom: Scheme presents the connectivity patterns under placebo and pramipexole.

It is interesting to note that an imbalanced network of reward expectation may also account for the tendency of adolescents to conduct risky behaviors and to make suboptimal decisions (Galvan et al., 2006; Casey et al., 2008; Van Leijenhorst et al., 2009). Adolescents are endowed with a functionally mature limbic system which is sensitive to incoming rewards, but it is well documented that the prefrontal cortex continues to develop into early adulthood. Consequently, as compared to young adults, adolescents demonstrated more activations in the VS and the insular cortex (Van Leijenhorst et al., 2009), but less activation in the prefrontal cortex during the anticipation of monetary rewards (Galvan et al., 2006). In other words, reward evaluation in adolescents is biased by the limbic system rather than the prefrontal system. The engagement of two systems has also been proposed to underlie decisions involving tradeoffs among benefits according to their expected delays (intertemporal choice). Choices of immediately available rewards are mediated by the activity of limbic regions, while choices of long-term rewards are supported by the activity of prefrontal regions (McClure et al., 2004).

Reward and addictive behavior

As pointed out above, a key question with regard to the role of the core reward areas VTA and VS in different behavioral contexts is how they might interface with different parts of the wider system shown in Figure 1 in these contexts. In the case of addiction, besides the expectation and delivery of the drug, we can also distinguish craving states, which induce active drug seeking and that can be elicited by drug-related cues. The investigation of connectivity patterns in addiction might be especially promising as it has been proposed on the basis of animal studies that there is a profound change in the way the ventral and dorsal striatum interact. Whereas drug-seeking behavior in the early phases of addiction is a goal-directed behavior with the drug being ingested because of its rewarding effects (similar to reward expectation described in Section “A Network Supporting Reward Expectation”), its behavior is maintained by drug-associated cues in the sense of a stimulus-response habit (Everitt et al., 2001; Redish, 2004; Everitt and Robbins, 2005; Volkow et al., 2006). In the initial phases drug seeking is thought to be controlled by the VS. Subsequently, control is gradually shifted to the dorsal striatum. This shift may be realized by serial “spiralling” connections between the NAcc and the dorsal striatum via midbrain dopamine neurons. In a recent lesion experiment Belin and Everitt (2008) used an intrastriatal disconnection procedure to disrupt this striato-midbrain-striatal connectivity bilaterally and found a greatly decreased drug-seeking behavior in rats addicted to cocaine. The shift from ventral to dorsal striatum in drug seeking behavior is consistent with evidence from MRI studies in addicted but drug-free humans in whom cue-elicited craving activates mainly the dorsal striatum as well as the amygdala and limbic prefrontal cortical areas (Grant et al., 1996; Childress et al., 1999; Garavan et al., 2000; Volkow et al., 2002) but not the VS.

These animal studies provide an interesting hypothesis regarding the connectivity of ventral and dorsal striatal regions in different stages of addiction. Thus far there are only few studies that have used connectivity measures in relation to addiction. Recently, Filbey et al. (2008) used alcoholic tastes that were delivered to heavy drinking volunteers. A region of interest (ROI) approach was used and connectivity was studied by performing correlations between the different ROIs. Unfortunately, the authors defined a large ROI encompassing both, ventral and dorsal striatum, in addition to ROIs encompassing the SN/VTA, the mPFC and the OFC. Significant correlations were reported between these regions when comparing alcohol-related cues vs. rest. Obviously, to test the question of a shift in connectivity from early to late phases of addiction would require the investigation of carefully selected participants and moreover the application of connectivity methods that are sensitive to the direction of information flow (see Section “Functional Connectivity Measures”).

There have been suggestions that obesity and the associated food intake behavior has strong parallels with drug addiction (Volkow and Wise, 2005; Volkow et al., 2008). For example, damage to the VTA–VS dopamine system suppresses free feeding and the willingness to press a lever for food rewards in rats. The same procedure also attenuates the reward effects of drugs (Wise and Rompre, 1989). Against this background, Stoeckel et al. (2009) recently investigated effective connectivity within a “reward-network” in obese and normal weight women who were exposed to pictures depicting high and low calorie food. Based on prior hypotheses they selected the NAcc, the amygdala, and the OFC and performed structural equation modeling. Compared to the normal weight women, the obese group showed less amygdala-modulated activation of the OFC and the NAcc. On the other hand overweight participants showed an increased influence of the OFC on the activation of the NAcc. These findings suggest that obese women not only show an overall greater activation of the reward system to food stimuli (as demonstrated in Stoeckel et al., 2008, using univariate analyses) but also showed differences in the interregional interaction in the studied network. While the analysis provided information regarding the direction of information flow between the brain areas, it has to be acknowledged that the model used in Stoeckel et al. (2009) was very simple due to the inherent limitations of the statistical analysis. For example, key areas involved in the motivational circuit outlined in Figure 1 and involved in the homeostatic regulation of food intake (e.g., the hypothalamus) were not included in the model. Another recent study used psychophysiological interaction analysis to investigate the differences in connectivity between appetizing and bland food stimuli (pictures) (Passamonti et al., 2009). Moreover, in a second step, the authors investigated to what extent these interactions were modulated by the external food sensitivity of the participants, i.e., their proneness to react to appetizing food. High external food sensitivity was associated with reduced differential connectivity in the network comprising the NAcc, amygdala, mPFC, and premotor cortical areas. This network has been suggested to be involved when controlling feeding behavior in animals (Kelley et al., 2005).

Concluding Remarks

The human brain roughly features 1011 neurons and 1014 synapses. The very architecture of the brain readily suggests that neurons act in coordinated concert on a microscopic level, e.g., within nuclei and cortical columns, and on a macroscopic regions, i.e., between distant brain regions. This fact has been largely neglected during the first two decades of brain imaging mostly for the lack of appropriate techniques. In the present paper we illustrated the advantages of connectivity analyses for the investigation of reward processing. What emerges at this point is still a very sketchy picture, however. What is needed is a more systematic assessment of the connectivity of the VTA–VS reward system in different contexts using the same methods. As illustrated by several examples in this review, altered connectivity of the VTA–VS system with other brain area may underlie behavioral and brain imaging changes observed during development and in pathological conditions. Such analyses will therefore contribute to our understanding of the pathophysiology of such states.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Key Concept

Rewards

Rewards are events and objects that modulate behavior: a behavior leading to a reward is more likely to occur again, whereas negative consequences lead to the opposite effects. Rewards are therefore potent learning signals.

Functional connectivity

Functional connectivity is defined as the statistical association or dependency among two or more neurophysiological time-series recorded in spatially remote areas. Correlational approaches are used and the direction of information flow can not be determined.

Effective connectivity

Effective connectivity approaches seek to describe causal influences of one brain area over another. Thus, the direction of information flow is determined. Such approaches need specified models with only a limited number of nodes.

Motivational network

This network of brain areas supports behaviors needed for survival. It produces spontaneous locomotor activity and explorative behavior in order to seek specific stimuli as well as ingestive, defensive and reproductive behaviors.

Structural connectivity

Structural connectivity describes anatomical connections between remote brain areas. Recent advances in neuroimaging (DTI-based tractography) allow to image white matter connections between areas.

Biography

Thomas F. Münte is Professor of Neuropsychology at the University of Magdeburg. Following his medical studies in Göttingen, Germany, he trained as a Neurologist at the Medical School Hannover, Germany. During the past 25 years he has conducted research on many different topics in cognitive neuroscience, ranging from early work on visual attention to bilingual language processing. Among his recent interests is the neural implementation of executive and action monitoring processes. In addition to his research, he is active as a clinical neurologist.