Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Mar 16.
Published in final edited form as: Neuron. 2016 Mar 3;89(6):1331–1342. doi: 10.1016/j.neuron.2016.02.002

Cognitive Neurostimulation: Learning to Volitionally Sustain Ventral Tegmental Area Activation

Jeff J MacInnes 1,2,7, Kathryn C Dickerson 1,2,7, Nan-kuei Chen 3,4, R Alison Adcock 1,2,5,6,*
PMCID: PMC5074682  NIHMSID: NIHMS810940  PMID: 26948894

SUMMARY

Activation of the ventral tegmental area (VTA) and mesolimbic networks is essential to motivation, performance, and learning. Humans routinely attempt to motivate themselves, with unclear efficacy or impact on VTA networks. Using fMRI, we found untrained participants’ motivational strategies failed to consistently activate VTA. After real-time VTA neurofeedback training, however, participants volitionally induced VTA activation without external aids, relative to baseline, Pre-Test, and control groups. VTA self-activation was accompanied by increased mesolimbic network connectivity. Among two comparison groups (no neurofeedback, false neurofeedback) and an alternate neurofeedback group (nucleus accumbens), none sustained activation in target regions of interest nor increased VTA functional connectivity. The results comprise two novel demonstrations: learning and generalization after VTA neurofeedback training and the ability to sustain VTA activation without external reward or reward cues. These findings suggest theoretical alignment of ideas about motivation and midbrain physiology and the potential for generalizable interventions to improve performance and learning.

INTRODUCTION

Adaptive behavior in humans and other animals depends on neuromodulatory neurotransmitter systems; these systems originate in small subcortical nuclei and project widely throughout the brain to modulate neuronal physiology and plasticity (Braver et al., 2014; Marder, 2012). Because they are evolutionarily ancient and architecturally rudimentary, neuromodulatory systems are often studied in the context of reflexive behaviors that are triggered by external stimuli and typically conceptualized as operating outside volitional control. Yet one among these—the mesolimbic dopamine system—is centrally and causally implicated not only in motivated behavior triggered by external cues but also in volition itself (Jahanshahi, 1998; Salamone and Correa, 2012).

Substantial prior empirical work has implicated the dopaminergic midbrain and its mesolimbic projections in a broad array of functional components of volitional behavior, including motivation (Salamone and Correa, 2012), valuation (Wimmer et al., 2012), action contingency (Tricomi et al., 2004), agency (Leotti and Delgado, 2011), effort allocation (Botvinick et al., 2009; Hall et al., 2001), response vigor (Beierholm et al., 2013; Niv et al., 2007), cognitive control (Braver and Barch, 2002), and action initiation (Nishino et al., 1987; Roitman et al., 2004; Stuber et al., 2005). If volitional motivated behavior is a primary function of the mesolimbic dopamine system, it follows that volitionally engendering motivation would engage the system’s source, the ventral tegmental area (VTA). This prediction, however, remains unproven. Although invasive stimulation and pharmacological challenges have implicated dopamine in motivated behavior (Salamone and Correa, 2012), research on physiological properties of midbrain neurons has focused instead on transient responses to external incentives and their cues (reviewed by Dayan and Niv, 2008; Montague et al., 2004; Schultz, 2007). Thus, despite strong implications from these complementary literatures, it remains uncertain whether volitional engagement of motivation is linked to physiological activation of the dopaminergic midbrain.

The uncertainty remains, in part, because experimental investigations, even in humans, have often employed reward cues or other external stimuli that confound claims about how internal representations, states, or intentions could account for activation. To demonstrate that activation is volitional or internally generated, it must occur independent of external stimuli (e.g., reward-associated cues) that are known to drive the VTA; this challenging criterion has not been met in prior studies of midbrain physiology in humans or animals (Adcock et al., 2006; Ballard et al., 2011; D’Ardenne et al., 2008; Fiorillo et al., 2003; Schultz, 2007; Sulzer et al., 2013; Takahashi et al., 2011; Wittmann et al., 2005).

We reasoned that humans could learn to use volitional cognitive strategies (e.g., self-generated subjective motivational states) to activate the VTA, without external reward cues or feedback. We predicted that volitional cognitive strategies—distinct from responses to external incentives—would correspond to a sustained temporal profile of activation, paralleling non-transient changes in animals during anticipation of extrinsic reward and cues (Fiorillo et al., 2003; Howe et al., 2013; Totah et al., 2013). Blood oxygen level-dependent (BOLD) fMRI signal increases in the midbrain, including VTA, have been shown to correlate with striatal dopamine release (Schott et al., 2008). Thus, the ability to strategically engage the VTA, as indexed by fMRI, could provide ecologically valid, temporally precise neuromodulation in mesolimbic efferent targets such as the nucleus accumbens (NAcc) and hippocampus (HPC), without pharmacological side effects or invasive direct stimulation. We use “cognitive neurostimulation” to refer to activating neuromodulatory source nuclei using only thoughts and imagery, without external aids.

Here, we used fMRI to address five critical questions about cognitive neurostimulation of the VTA: (1) Will intuitively generating subjective motivational states, without the help of reward cues, consistently activate VTA? (2) Are the dynamics of VTA activation during cognitive neurostimulation accounted for by transient responses to external events or instead consistent with sustained internal states? (3) Can participants use neurofeedback from VTA to learn to achieve consistent cognitive neurostimulation after training? (4) Will the ability to achieve cognitive neurostimulation of VTA impact broader neural systems, for example, untrained targets of mesolimbic efferents? (5) To the extent that VTA self-activation is possible, does it rely on veridical neurofeedback training, or can alternate training procedures suffice?

We developed a paradigm in which participants were instructed to “generate a heightened state of motivation” during Pre-test, Training, and Post-test periods. To our primary experimental group, we provided veridical real-time neurofeedback from the VTA. Additional groups were included in which the nature of the feedback was manipulated; one group was provided a dynamic cue to help maintain attention while another group received false feedback (FF; noise). Furthermore, we included an alternative feedback group, who received veridical neurofeedback from the NAcc. Rather than simple operant conditioning, we encouraged meta-cognitive strategies to foster generalization of learning beyond the training. Transient “reward responses” were avoided by using habituated instructional cues with no prior reward association. To prevent carryover of reward associations from Training, cues changed between Training and Tests. Importantly, the Pre-test and Post-test did not contain reward cues, dynamic attentional aids, or neurofeedback.

Our primary analyses aimed to demonstrate reliable VTA activation that was uncorrelated with transient external events, including neurofeedback. We focused analyses on Post-test activation both because of the theoretical importance of demonstrating volitional VTA activation independent of external reward and because of the implications of demonstrating learning from neurofeedback. In particular, the real-world impact of an effect that is evident only during neurofeedback training is severely limited, whereas if an intervention can be learned and generalized, it can potentially be implemented in daily life. Notably, the pervasive pattern of findings in extant neurofeedback literature has been a change in neural responsivity to feedback during training, with no evidence of learning or generalization to novel contexts; indeed, prior efforts to train reward systems follow this pattern (Greer et al., 2014; Sulzer et al., 2013).

Specifically, fMRI analyses aimed to test the following hypotheses: (1) instruction to engender motivation volitionally without reward cues would not reliably activate the VTA prior to training, (2) neurofeedback training would help participants learn to induce activation consistently, as evidenced by Post-test activation, (3) volitional VTA activation without reward cues (i.e., cognitive neurostimulation) would induce a sustained rather than a cue-evoked or transient VTA signal, and (4) effective cognitive neurostimulation of VTA would increase connectivity with untrained mesolimbic networks.

RESULTS

Seventy-three participants (VTA Feedback, n = 19; Visual Control [VC], n = 20; NAcc Feedback, n = 20; and Noise FF, n = 14) completed a Pre-test run, three Training runs, and a Post-test run during fMRI scanning (Figure 1). On COUNT baseline trials, all groups counted backward. On ACTIVATE trials, participants were instructed to “generate a heightened state of motivation” (see Participant Instruction in Supplemental Experimental Procedures for instruction scripts). Examples of possible strategies were provided, including encouraging phrases (“You can do it!”) and imagery (“Crossing the finish line of a marathon”). (Table S1 presents subjective ratings by group.) Importantly, however, participants were explicitly encouraged to identify personally relevant strategies, monitor their efficacy through out the study, and update them as needed, selecting the most effective for use in the Post-test. Participants were randomly assigned to groups; the critical group difference was in the information provided by a fluctuating thermometer during training. For the VTA Feedback group, the thermometer represented the magnitude of VTA BOLD signal, updated every second. For the FF group, the thermometer represented noise values randomized around zero. For the NAcc Feedback group, the thermometer represented NAcc BOLD signal. Participants in each of the three feedback groups attempted to increase the level of the thermometer to exceed a target height (see Figure 1). For the VC group, the thermometer moved incrementally along a fixed pattern, unrelated to brain activity, controlling for attentional focus while participants increased motivation. To assess participants’ ability to self-activate regions within the dopaminergic network, we first compared the effects of veridical VTA neurofeedback to a VC group and noise false neurofeedback (FF group). Then, to determine whether our findings were generalizable to other mesolimbic regions, we compared the effects of veridical NAcc neurofeedback to the same VC and FF groups (see Supplemental Information for a direct comparison of the VTA and NAcc neurofeedback groups).

Figure 1. Task Design.

Figure 1

Open in a new tab

Pre-test and Post-test: all groups completed identical test runs. During ACTIVATE trials, participants tried to increase motivation using only internally generated thoughts and imagery, without reward cues or rt-fMRI neurofeedback. During COUNT baseline trials, participants counted backward. Training: during ACTIVATE trials, participants in VTA and NAcc Feedback groups tried to increase motivation and received veridical neurofeedback from either VTA or NAcc. FF participants received noise neurofeedback they were told was veridical. VC participants viewed predictable patterns indicating the duration of the ACTIVATE period. During REST trials, each group’s thermometer display presented a random (VTA Feedback, NaCC Feedback, FF groups) or predictable (VC group) pattern. COUNT trials were identical across all runs. An inter-trial interval ranging from 3.5–5.5 s separated all trials.

fMRI Results

Primary analyses focused on a priori regions of interest (ROIs): dopaminergic midbrain centered on the VTA (from the VTA probabilistic atlas we developed; Ballard et al., 2011; Murty et al., 2014) and the NAcc (from Greer et al., 2014) in the contrast of ACTIVATE versus COUNT trials. As noted, we used a probabilistic atlas to define the VTA, but we cannot rule out engagement of other nearby midbrain regions, including the substantia nigra (SN). Future work using high-resolution fMRI and other techniques will explore the specificity of our findings. (For whole-volume results, see Figures S1 and S2; see Supplemental Information for additional control analyses of ACTIVATE versus REST).

Do Untrained Participants Attempting Volitional Motivation Activate the VTA?

We hypothesized no group differences, prior to training, in the ability to increase VTA activation. As predicted, there was no significant main effect of group (F(2,50) = 0.48, p > 0.1) in the Pretest, as shown in a repeated-measures ANOVA using time point (1–20) as a within-subjects and group (VTA, VC, FF) as a between-subjects factor. There was a main effect of time point (F(16.79,839.37) = 4.81, p < 0.001; note that degrees of freedom reflect correction for sphericity violation), but no significant interaction (F(33.58,839.37) = 1.00, p > 0.1) (see Figures 2 and 3). Furthermore, no group showed reliable changes in VTA activation compared with baseline (VTA Feedback: t(18) = 0.48, p > 0.1; VC: t(19) = 1.51, p > 0.1; FF: t(13) = 2.22, p = 0.04—non-significant deactivation after sequential Bonferroni correction). In summary, prior to training, no group showed reliable VTA activation, nor did the groups significantly differ from each other.

Figure 2. Significant VTA Activation and Group Differences Emerged in Post-test following Feedback Training.

Figure 2

Open in a new tab

(A) VTA ROI defined in an independent sample of 50 participants. Color scale denotes probabilistic weighting of the ROI.

(B) Test run × group interaction plot (p < 0.05) representing percentage signal difference for mean ACTIVATE > COUNT values. Pre-test: no significant activations or group differences. Post-test: VTA Feedback group self-activated the VTA relative to baseline (p < 0.005) and to Control (p < 0.0005) and FF (p < 0.05) groups.

Figure 3. Consistent VTA Activation and Group Differences Emerged with Feedback Training.

Figure 3

Open in a new tab

ERA time courses for ACTIVATE >COUNT during Test and Training trials. Waveforms represent percentage signal difference from baseline (shading, ± SEM). The time course for both ACTIVATE and COUNT is calculated relative to the preceding 3-s inter-trial interval. To compare the time series, we subtracted COUNT from the ACTIVATE time series. Time courses were segmented at 10 s to examine sustained activation (solid horizontal bars represent means). Pre-test: no significant positive activations or group differences. Training: VTA Feedback group showed greater VTA activation than the VC group in both early (p < 0.0001) and late phases of trials (p < 0.05; i.e., across the entire 20 s), but did not significantly differ from FF group (p > 0.1). Post-test: the VTA Feedback group sustained greater activation relative to baseline (early, late, and overall p < 0.05), relative to the VC group (early, late, and overall p < 0.005), and relative to FF group (late and overall p < 0.05). Post hoc t tests (p < 0.05) are denoted by the keys below the time courses. Center white circle, baseline; orange, VTA Feedback; blue, VC; gray, FF; black line, a significant difference.

Is VTA Activation Increased during Neurofeedback Training?

We examined activation time courses for each Training run and calculated group event-related averages (ERAs; relative to 3-s pre-trial baseline) to test for group differences. VTA activation significantly differed across groups, as evidenced by a main effect of group (F(2,50) = 5.03, p < 0.05) on a 3 (run: Training 1, 2, 3) × 3 (group: VTA Feedback, VC, FF) × 20 (time point: 1–20) ANOVA. VTA activation was greater for the VTA compared with the VC group (t(37) = 3.82, p < 0.0005). The FF group did not differ from either the VTA (t(31) = 1.18, p > 0.1) or VC (t(18.22) = 1.36, p > 0.1) groups. There was also a main effect of time point (F(12.49, 624.72) = 5.84, p < 0.001), but no effect of run (F(2, 100) = 0.00, p > 0.1) and no significant interactions (Ps > 0.1). Thus, relative to the VC group, the VTA feedback group showed enhanced activation over the duration of the ACTIVATE trial (Figure 3).

What Are the Temporal Dynamics of VTA Responses during Neurofeedback?

As 95% of the BOLD signal from a transient response decays within 10 s (Huettel et al., 2009), we bisected each trial into 10-s phases to delineate transient from sustained properties to the VTA response. We performed fully corrected, post hoc comparisons examining activation in the early and late phases of the trial. Relative to baseline, the VTA Feedback group increased activation in the first half of the trial (t(18) = 4.74, p < 0.0005). Interestingly, during neurofeedback training, there were no significant differences between VTA BOLD signal in the VTA and FF groups in the first (t(31) = 1.36, p > 0.1) or second half (t(31) = 0.89, p > 0.1) of the trial. The FF group also did not significantly differ from the VC group (first half (t(19.64) = 2.09, p = 0.05; non-significant after sequential Bonferroni correction), second half (t(32) = 0.70, p > 0.1). VTA Feedback group activation was significantly greater than the VC group during both the first and second halves of the trial (Figure 3, middle; early: (t(37) = 4.77, p < 0.0001; late: t(37) = 2.13, p < 0.05).

Is VTA Neurofeedback Sufficient, and Necessary, to Learning VTA Self-Activation?

To examine learning after neurofeedback training, we asked groups to use their best cognitive strategies to attempt cognitive neurostimulation, that is, to induce VTA activation without feedback or reward cues. We compared differences in activation across runs and groups using a 2 (run: Pre-test, Post-test) × 3 (group: VTA Feedback, VC, FF) × 20 (time point: 1–20) ANOVA. Main effects of group (F(2, 50) = 5.46, p < 0.01), time point (F(14.62, 730.98) = 3.98, p < 0.001), and a marginally significant main effect of run (F(1, 50) = 2.98, p = 0.09) were observed, with significant run × group (F(2, 50) = 3.26, p < 0.05) and run × time point interactions (F(15.69, 784.24) = 1.72, p < 0.05); there were no other significant interactions (p > 0.1) (Figures 2 and 3). The interaction of run × group reflected no group differences in the Pre-Test in either phase (early: VTA Feedback versus VC (t(37) = 0.20, p > 0.1), VTA Feedback versus FF (t(31) = 0.34, p > 0.1), VC versus FF (t(32) = 0.64, p > 0.1); late: VTA Feedback versus VC (t(28.90) = 0.96, p > 0.1), VTA Feedback versus FF (t(31) = 1.26, p > 0.1), VC versus FF (t(32) = 0.64, p > 0.1) but did indicate greater VTA activation in the Post-test for the VTA Feedback compared with both control groups (early: VTA Feedback versus VC [t(37)= 3.06, p <0.005], VTA Feedback versus FF [t(31) = 1.72, p = 0.10], VC versus FF [t(32) = 1.01, p > 0.1]; late: VTA Feedback versus VC [t(37) = 4.07, p < 0.0005], VTA Feedback versus FF [t(31) = 2.62, p < 0.05], VC versus FF [t(32) = 1.01, p > 0.1]; overall: VTA Feedback versus VC [t(37) = 3.87, p < 0.0005], VTA Feedback versus FF [t(31) = 2.48, p < 0.05], VC versus FF [t(32) = 1.09, p > 0.1]).

In addition to group differences, VTA Feedback group activation at Post-test was significantly greater than Pre-test (t(18) = 2.36, p < 0.05) and greater than baseline (early: t(18) = 2.88, p < 0.05; late: t(18) = 3.29, p < 0.005; overall: t(18) = 3.52, p < 0.005). In contrast, neither control group activated the VTA significantly greater than baseline (for the VC group, early: t(19) = 1.56, p > 0.1, late: t(19) = 2.45, p = 0.02 [non-significant deactivation after sequential Bonferroni correction], overall: t(19) = 2.08, p = 0.05 [non-significant deactivation after sequential Bonferroni correction]; FF group, early: t(13) = 0.02, p > 0.1, late: t(13) = 0.66, p > 0.1, overall: t(13) = 0.36, p > 0.1).

Do Untrained Participants Attempting Volitional Motivation Activate the NAcc?

We hypothesized no group differences in the ability to increase NAcc activation prior to receiving neurofeedback. As predicted, the groups did not differ as revealed by no significant main effect of group (F(2,51) = 2.40, p = 0.10) on a repeated-measures ANOVA using time point (1–20) as a within-subjects factor and group (NAcc, VC, FF) as a between-subjects factor. There was a main effect of time point (F(16.04,817.88) = 5.08, p < 0.001), but no significant interaction (F(32.07,817.88) = 1.00, p > 0.1) (see Figures 4 and 5). Furthermore, no group showed reliable NAcc activation above baseline (NAcc: t(19) = 0.07, p > 0.1; both the VC and FF groups were significantly below baseline, t(19) = 2.81, p < 0.05 and [t(13) = 2.37, p < 0.05], respectively). In summary, prior to training, no group showed reliable NAcc activation, and the groups did not significantly differ.

Figure 4. No Significant NAcc Activation or Group Differences in Test Runs.

Figure 4

Open in a new tab

(A) NAcc ROI defined by Greer et al. (2014).

(B) Non-significant test run × group interaction plot (p > 0.1) representing percentage signal difference for mean ACTIVATE > COUNT values. Pre-test: No significant corrected positive activations or group differences were observed. Both control groups were significantly deactivated relative to baseline (p < 0.05). Post-test: The groups did not significantly differ from each other (p ≥ 0.09) and no group self-activated the NAcc relative to baseline (p ≥ 0.1).

Figure 5. No Significant NAcc Activation or Group Differences prior to, during, or following Feedback Training.

Figure 5

Open in a new tab

ERAs for ACTIVATE > COUNT during Test and Training trials. Waveforms represent percentage signal difference from baseline (shading, ± SEM). The time course for both ACTIVATE and COUNT is calculated relative to the preceding 3-s inter-trial interval. To compare the time series, we subtracted COUNT from the ACTIVATE time series. Time courses were segmented at 10 s to examine sustained activation (solid horizontal bars represent means). Pre-test: no significant corrected activations or group differences. Training: no significant activations or group differences. Post-test: no significant activations or group differences. Significant mean differences from baseline (p < 0.05) are denoted by the keys below the time courses. Center white circle, baseline; green, NAcc Feedback; blue, VC; gray, FF; black line, a significant difference.

Is NAcc Activation Increased during Neurofeedback?

NAcc activation was not significantly greater in the NAcc Feedback relative to the other groups, evidenced by no significant main effect of group (F(2,51) = 0.28, p > 0.1) in a 3 (run: Training 1, 2, 3) × 20 (time point: 1–20) × 3 (group: NAcc, VC, FF) ANOVA. There was a main effect of time point (F(10.69,545.03) = 7.10, p < 0.001), but no main effect of run (F(2,102) = 0.28, p > 0.1) or significant interactions (p > 0.13). Furthermore, no group successfully increased NAcc activation above baseline during training (NAcc: t(19) = 0.29, p > 0.1, VC: t(19) = 0.62, p > 0.1, FF: t(13) = 0.59, p > 0.1).

Does NAcc Neurofeedback Training Confer Learning?

We compared differences in activation across runs and groups using a 2 (run: Pre-Test, Post-Test) × 3 (group: NAcc Feedback, VC, FF) × 20 (time point: 1–20) ANOVA. A trend-level main effect of group (F(2,51) = 2.98, p = 0.06), significant main effect of run (F(1,51) = 6.14, p < 0.05), and significant main effect of time point (F(13.15,670.53) = 5.55, p < 0.001) were observed, with no significant interactions (p > 0.11). The trend-level main effect of group was driven by marginally significant greater activation for the NAcc group than VC group in the Pre-test (t(38) = 1.99, p = 0.05), but no significant differences in the Post-test (t(38) = 0.84, p > 0.1). Furthermore, no group increased NAcc activation relative to baseline (NAcc group, early: t(19) = 1.74, p = 0.1, late: t(19) = 0.47, p > 0.1; VC, early: t(19) = 0.29, p > 0.1, late: t(19) = 0.02, p > 0.1; FF, early: (t(13) = 0.62, p > 0.1, late: t(13) = 1.44, p > 0.1). Thus, the NAcc Feedback group did not demonstrate learned NAcc activation. (There was also no significant VTA activation in the NAcc Feedback group; see Figure S4.)

Does Mesolimbic Neurofeedback Training Change Network Connectivity?

We hypothesized that learned cognitive neurostimulation of the VTA in the Post-test would increase connectivity with mesolimbic networks, including the HPC and striatum. Since we observed no sustained NAcc activation following NAcc neurofeedback training (unlike sustained VTA activation following VTA training), we predicted increased mesolimbic functional connectivity would be specific to VTA neurofeedback training. We used a functional connectivity analysis to investigate changes in connectivity in the Post-test compared with the Pre-test, between the following ROIs: VTA, NAcc, bilateral HPC, and bilateral caudate nucleus. Following VTA neurofeedback training, Z-scored changes in Pearson’s correlation coefficients revealed significant increased functional connectivity between the VTA and bilateral HPC, as well as between the NAcc and the bilateral HPC (Figure 6; see Table S2 for statistics). There were no significant changes in connectivity in this network following NAcc neurofeedback training.

Figure 6. Functional Connectivity Significantly Increased in Mesolimbic Networks following VTA, but Not NAcc, Feedback.

Figure 6

Open in a new tab

In the VTA Feedback group (left), both the VTA and the NAcc ROIs exhibited significantly greater Pre-test to Post-test connectivity with the bilateral HPC (p < 0.05). There were no significant connectivity changes for the NAcc Feedback group (p > 0.1; right), resulting in a significant Run × Group interaction for these ROIs (see Table S2 in the Supplemental Information). Line thickness denotes the change in correlation strength from the Pre-test to Post-test (Z scored). Line color indicates significant/non-significant changes in connectivity (dark/light gray). The line pattern indicates the direction of change in Z-scored r values (solid lines, increased connectivity from Pre-test to Post-test; dotted lines, decreased connectivity from Pre-test to Post-test).

We also examined changes in connectivity from the Pre-test to Training (see Table S3 for statistics). Significant changes in functional connectivity were again only observed in the VTA Feedback group. Interestingly, increased VTA-bilateral HPC connectivity occurred during Training and remained in the Post-test. Increased VTA-Left Caudate, and NAcc-Left Caudate connectivity was specific to the Training phase. In summary, increased midbrain to caudate connectivity occurred only during the presence of feedback (training), while increased HPC connectivity began with the VTA during Training and generalized to include both the VTA and NAcc during Post-test.

DISCUSSION

This study aimed to investigate activation of the dopaminergic midbrain (centered on the VTA) via volitional cognitive strategies without external reward cues or feedback. The findings include the following novel demonstrations. First, untrained participants were initially unable to use motivational strategies to consistently increase VTA or NAcc activation. Second, after rt-fMRI neuro-feedback training, as evidenced in the Post-test, the VTA group (and, notably, only this group) self-activated the VTA without novel stimuli, reward cues, or feedback. Third, the VTA Feedback group sustained VTA activation during the Post-test throughout the 20-s trial. Fourth, the two control groups and the alternate feedback group failed to achieve self-activation in the Post-test (see Supplemental Information). Fifth, the NAcc Feedback group failed to activate the NAcc during rt-fMRI neurofeedback training and following training in the Post-test (but see Supplemental Information for analysis method comparable with Greer et al., 2014). Sixth, the impact of VTA neurofeedback training extended beyond the VTA, resulting in increased functional connectivity throughout mesolimbic targets following training, whereas NAcc training produced no significant changes in mesolimbic functional connectivity.

Potential Mechanisms of Learning from Neurofeedback

Prior to training, participants’ self-generated motivational strategies failed to consistently drive VTA (or NAcc) activation; no group significantly activated the target ROI at Pre-test. In fact, some individuals in all groups showed deactivations, particularly during the late phase of the trial. Note that to support an inference of learning, critical analyses relied not only on significant activation above baseline and group differences in the Post-test, but also Pre-test to Post-test changes, to control for effects of random differences in Pre-test activation.

Notably, neurofeedback training proved to be critical for learning to elicit VTA activation without external aids. The null Pre-test results show a disconnect between participants’ initial response to our instructions and engagement of the biological systems theorized to underlie motivated behaviors. Our findings suggest that veridical neurofeedback training demonstrated the link between participant’s internally generated thoughts and imagery and midbrain activation and that by virtue of this link, participants were able to select and subsequently exploit strategies that increased VTA activation. Individuals in the VC, FF, and NAcc Feedback groups were not successful at sustaining VTA activation relative to baseline in the Post-test; thus, we conclude that in the absence of an accurate index of midbrain activation, these participants were unable to appropriately select strategies to maximize VTA responses.

The question arises whether some characteristic of the feedback experience, rather than the participants’ internal representations of motivated states, helped them sustain VTA activation. This question is especially pertinent given the task demand of working to achieve a goal (e.g., raising the thermometer) conforms to most conceptions of motivation. The lack of evidence for enhanced activation in the Post-test in the FF and NAcc groups suggests that merely having an objective goal during the training phase is not enough to account for Post-test VTA activation. In addition, positive feedback itself is expected to drive VTA activation, an important confound not addressed in prior studies, which have reported only activation during training (Sulzer et al., 2013). We addressed this concern with control analyses modeling the effect of moment-to-moment VTA feedback values (thermometer height) as well as the total amount of positive feedback displayed on the thermometer during training; neither of these accounted for VTA activation during Training or Test (see Supplemental Information). Additionally, we included groups that received noise feedback (FF group) and veridical NAcc feedback. The VTA Feedback group received more positive feedback than the FF group. (By design, the FF group had a mean thermometer height of 0. This was also true of the VC group, but they were instructed, and believed, that the signal was unrelated to brain activity.) Interestingly, the amount of positive feedback received during training for the VTA Feedback and NAcc Feedback groups did not significantly differ (t(37) = 1.64, p > 0.1). Finally, the FF group demonstrated increased VTA activation during Training, but not in the Post-test. Together, these data suggest that Training-phase activation includes multiple processes not all of which contributed to Post-test self-activation and that receiving veridical VTA feedback was important for learning to self-activate the VTA in the Post-test. The confounds posed by engaging with neurofeedback highlight the importance of the current finding, where generalization to a Post-test occurred without neurofeedback or reward cues.

How did our VTA neurofeedback participants sustain VTA activation in the Post-test? Our interpretation is that participants reactivated states cultivated during training; it is an open question whether these states reflected simulations of the tangible goals present during training or specific strategies used to achieve VTA activation. One way to answer this would be to ask participants to generate VTA activation without suggesting any strategies. However, extant studies of self-guided neurofeedback require heroic regimes of thousands of training trials over many days (e.g., Shibata et al., 2011), making null results highly vulnerable to false negatives and introducing a difficulty confound for comparison with our twenty-minute training protocol. An alternative would be to employ different instructions with the same feedback device. Indeed, the one prior study attempting dopaminergic midbrain regulation used instructions related to reward and positive affect combined with external reward cues; that study reported neither increased Post-test activation nor sustained signal in midbrain (Sulzer et al., 2013).

Relationship to Ideas about Motivation and Dopamine

Our data address current theoretical conceptions about dopamine physiology in two ways. First, we show that although untrained participants’ intuitions about how to do so may be unreliable, volitional motivational strategies can be harnessed to produce VTA activation. The states used to drive VTA activation here were internally generated, complementing the corpus of research on midbrain responses to external stimuli (Adcock et al., 2006; Ballard et al., 2011; D’Ardenne et al., 2008; Fiorillo et al., 2003; Schultz, 2007; Sulzer et al., 2013; Takahashi et al., 2011; Wittmann et al., 2005). While it has been theorized that midbrain dopamine is the biological substrate of all forms of motivation (Bromberg-Martin et al., 2010; Salamone and Correa, 2012), only a few studies have examined motivation not driven by extrinsic incentives, i.e., intrinsic motivation, or behavior pursued due to inherent pleasure in an activity (Lee and Reeve, 2013; Lee et al., 2012; Murayama et al., 2010). The motivational states we examined did not require extrinsic incentives, but should also be distinguished from intrinsic motivation, as they were not an expression of inherent anticipated pleasure in the task. To distinguish it from motivated states arising from reward cues (extrinsic or intrinsic), tasks, or external stimuli, we refer to this state as “volitional motivation.”

Second, we show that healthy individuals can learn to sustain VTA activation over a period of 20 s. This timescale is novel in the human literature and may relate to a controversial finding in the animal literature (Beeler et al., 2010; Fiorillo et al., 2003; Nishino et al., 1987; Niv et al., 2005; Schultz, 2007). Although a dichotomized view of dopamine neuron physiology as either tonic or phasic has guided the field for many years, converging evidence across methodologies (Fiorillo et al., 2003; Howe et al., 2013; Totah et al., 2013) suggests a third type of dopamine response profile: a sustained, ramping signal evident during anticipation of reward. This ramping signal is distinct from rapid phasic responses, but also differs from tonic signals theorized to reflect a summation of prior phasic events (Niv et al., 2007) or spontaneous firing (Goto et al., 2007); instead, it appears more consistent with sustained excitatory inputs. The VTA activations observed here are inconsistent with transient neural responses to external events and may be consistent with the sustained dopamine profile described in the animal work. While our VTA BOLD signal did not demonstrate a ramping profile, the sustained nature of the signal converges with the neuronal signal observed in the animal work, in that both are novel profiles not accounted for by transient signals. Alternatively, it is also possible our results reflect summed sequential phasic responses as participants refresh strategies; BOLD imaging is currently unable to resolve this question. More fundamentally, however, these sustained signals were observed in the absence of external reward cues and therefore must have been elicited through internal representations.

Neurofeedback training not only increased the ability to produce volitional VTA activation, it also increased connectivity between the VTA and bilateral HPC, as well as NAcc and bilateral HPC. These findings are consistent with excitation in monosynaptic efferents from the VTA to the HPC (Gasbarri et al., 1997) and from the HPC to the NAcc (Phillipson and Griffiths, 1985).

Relationship to Previous Work on Reward System Neurofeedback

Critically, we found the ability to induce VTA activation persisted in the Post-test, conducted without feedback, revealing learning. This is a rare outcome among neurofeedback studies and specifically distinguishes the current findings from prior attempts to train regulation of the midbrain (Sulzer et al., 2013) that reported changes in neural responsivity during feedback, but no evidence of learning or generalization to novel contexts. Our approach included several features that may explain this difference. First, our instructions emphasized employing motivational strategies, rather than rewarding mental imagery. Second, we used no external reward cues which confound attribution of activation to volitional or internal states. Third, due to its small size and proximity to signal-disrupting sinuses, imaging the VTA can be challenging with fMRI (D’Ardenne et al., 2008); to counteract these difficulties, we used an independently defined probabilistic VTA atlas and a short TR (1 s) to increase the number of samples and thus signal to noise. This protocol limited the number of slices we acquired to 18 (see Figure S3 for representative slice coverage), but allowed us to increase our ability to detect significant activation in the VTA, which was our primary objective. Fourth, rather than relying on model-free operant conditioning, we used an explicitly meta-cognitive instructional approach that we predicted would facilitate generalization outside training. We encouraged wide exploration of individualized motivational strategies, with selection of the most effective strategy to exploit in the Post-test. This strategy selection and exploitation, together with the removal of additional cognitive load from feedback processing, may explain why the Post-test activations are not reduced from those seen during training. Importantly, the Post-test activation comprises two novel demonstrations: first, learning and generalization after veridical VTA neurofeedback training, and second, volitionally sustained VTA activation without external reward or reward cues. These findings have crucial theoretical and real-world implications, including development of cognitive neurostimulation of neuromodulatory systems as clinical and research tools.

Unlike participants who received VTA neurofeedback, those who received veridical NAcc neurofeedback in our study were unable to produce any significant increases in NAcc activation or functional connectivity in the Post-test. The negative Post-test finding is consistent with findings reported by Greer et al. (2014). (Patterns of NAcc activation during feedback training were subtly different across these studies but were reconcilable on the basis of analytic and instruction differences; see control analyses, Supplemental Information.)

It should be noted that only the VTA Feedback group significantly activated the VTA in the Post-test. The NAcc Feedback group, despite showing no NAcc activation, showed non-significant VTA activation at Post-test (see Figure S4). As a result, VTA activation did not significantly differ between the VTA and NAcc Feedback groups in the Pre-test or Post-test. These VTA activations during NAcc feedback, although weaker, are unsurprising given the close functional connectivity between the NAcc and VTA. In sum, the pattern of results showed NAcc neurofeedback was not as effective as VTA neurofeedback in engaging mesolimbic networks and supporting self-activation at Post-test. These findings raise interesting questions, not only about underlying neural mechanisms, but also about potential effects of neurofeedback signal properties, for future work aimed at understanding how (and what) people learn from neurofeedback.

Limitations and Open Questions

The primary limitation of this study is that BOLD fMRI does not allow for the direct measurement of neurotransmitters; it is not a direct index of dopamine release. However, although the VTA contains non-dopamine neurons, BOLD activation of the VTA has been shown to predict dopamine release in the striatum (Schott et al., 2008). Future PET or pharmacological studies are required to confirm that cognitive neurostimulation of the VTA directly affects dopamine signaling. A second limitation is that our use of a probabilistic VTA atlas, which optimized detection and scan time efficiency, also limits claims of anatomical specificity to VTA. Of note, we recently demonstrated that our probabilistic atlases can effectively differentiate the VTA from the SN in resting state functional connectivity (Murty et al., 2014). While none of the conclusions herein rest on SN/VTA distinctions, future work may include specialized scans (e.g., proton density) in order to characterize unique versus shared contributions of VTA and SN to self-activation.

Our current design did not allow us to link learned VTA activation to a behavioral benefit. We nonetheless observed plasticity in network physiology during learned cognitive neurostimulation, namely increased functional connectivity to untrained mesolimbic efferent regions. Future studies will aim to demonstrate the implied behavioral impact of these physiological changes, leveraging volitional VTA activation to influence cognition and learning. It is critical to explore what exactly participants learn during training and how it may lead to enhanced, volitional VTA activation in the absence of neurofeedback in the Post-test. One approach could include instructional manipulations to bias participants toward rigid, operant conditioning versus flexible, meta-cognitive strategies. This will allow us to test our hypothesis that the generalization seen in the current study was supported by a more flexible, hippocampal-based learning mechanism, which contrasts with the striatal, feedback-based learning assumed in most neurofeedback studies. For development of efficient training paradigms, it is crucial to systematically study categories of strategies (e.g., emphasizing affective versus motor imagery) and individual differences to identify the most effective methods. Similarly, our future work will aim to identify baseline or dynamic predictors of improvement in diverse cohorts of participants.

Translational Implications

Our demonstration of cognitive neurostimulation of the VTA implies new approaches for research and practice in multiple fields. In particular, the transfer of skills to the Post-Test after brief neurofeedback training suggests potential transfer to real-world contexts for educational and clinical applications.

As the major source of dopamine, the VTA/SN project to diverse cortical and subcortical targets. Dopamine is widely implicated in mental health (reviewed in Maia and Frank, 2011; Montague et al., 2004) and adaptive behavior (reviewed in Braver et al., 2014; Salamone and Correa, 2012; Schultz, 2007; Shohamy and Adcock, 2010; Treadway et al., 2012). The range of behaviors dependent on dopamine neuromodulation is broad and far reaching (Alcaro et al., 2007). Potential benefits of learning to sustain dopamine release include increased perceptual signal to noise (Lou et al., 2011; Pessoa and Engelmann, 2010), invigoration of motor responses (Beierholm et al., 2013; Niv et al., 2007), improved attention (Volkow et al., 2009), working memory (Cools and D’Esposito, 2011; Goldman-Rakic, 1997), and long-term memory encoding (Lisman et al., 2011; Shohamy and Adcock, 2010). Cognitive neurostimulation could achieve some of these benefits with greater temporal precision and fewer side effects than chronic pharmacotherapy or deep brain stimulation, suggesting the potential for safer, more efficient interventions.

More specifically, prior work identifies the potential to enhance performance and learning by yoking increased dopamine release in the NAcc and HPC to appropriate contexts. If VTA-NAcc connectivity correlates with success, enhancing it could help individuals persevere in accomplishing difficult tasks (e.g., working on a challenging math problem; Volkow et al., 2011). Prior empirical work has already demonstrated that VTA-HPC connectivity predicts successful memory (Adcock et al., 2006; Callan and Schweighofer, 2008; Duncan et al., 2014; Wolosin et al., 2012); thus, the ability to volitionally produce states conducive to learning could enhance education and clinical outcomes in learning-based therapies. The finding that volitional VTA activation can be learned and accomplished without external aids after brief neurofeedback training establishes translation to these settings as a testable direct next step.

Conclusions

For centuries, scientists and philosophers have questioned how best to engender and sustain motivation (James, 1950; Plato, 1991). That a simple answer has yet to emerge should come as no surprise; our findings illuminate a gap between untrained subjective intuitions about motivational strategies and the ability to activate systems theorized to underlie motivated behavior. Yet, we found that with appropriate training, it is possible to learn to volitionally activate and sustain the VTA without external stimuli. These findings suggest new frameworks for aligning psychological with biological perspectives and for understanding and harnessing the power of neuromodulatory systems.

EXPERIMENTAL PROCEDURES

Participants

Ninety-seven participants provided informed consent in accordance with the Duke University Institutional Review Board. Twenty-four participants were excluded due to failure to follow instructions, excessive motion (>3 mm), sleeping in the scanner, medication exclusion, a score of >14 on the Beck Depression Inventory, or technical failures. All participants were right handed and were screened for contraindications to MRI, history of psychological or neurologic illness, and pregnancy. Nineteen VTA Feedback (9 female, mean age 24), 20 VC (10 female, mean age 23), 20 NAcc Feedback (16 female, mean age 23), and 14 Noise FF (9 female, mean age 21) participants were included in the final dataset.

fMRI Session

Scanning began with a 2-min resting-state scan for registration of the probabilistic VTA atlas and NAcc ROI (MNI space) to individual functional space. The task consisted of five runs: one Pre-test, three Training runs, and one Post-test.

fMRI Acquisition

A 3T GE MR750 MRI scanner was used to acquire BOLD images. Functional runs were collected using an echo-planar sequence with the parameters: TR, 1 s; TE, 28 ms; flip angle, 90°; voxel size, 3 × 3 × 3.8 mm; 18 oblique axial slices, parallel to the anterior commissure—posterior commissure axis (see Figure S3 for representative slice acquisition). The first eight volumes of each run were discarded to permit stabilization of the net magnetization. Fast spoiled gradient echo high-resolution whole-volume T1-weighted images (voxel size, 1 × 1 × 1 mm) were acquired. Physiological measures including raw cardiac and respiration signals were recorded using a pulse oximeter and a respiration belt.

Task and Instruction

All Groups

Participants were instructed on all task phases prior to entering the MRI. Following each run, participants completed task ratings (see Supplemental Information).

VTA Feedback and NAcc Feedback Groups

The task began with a Pre-test to measure participants’ baseline ability to self-activate the target ROI (VTA or NAcc) in the absence of neurofeedback or reward cues (Figure 1). Participants were presented with two trial types: COUNT and ACTIVATE. On COUNT trials participants counted backward from 300-in increments of four. On ACTIVATE trials participants tried to generate a state of heightened motivation. To avoid biasing participants’ strategies, we provided limited examples during instruction (e.g., “Try positive motivational phrases like ‘you can do it!’ or imagining personally relevant scenarios”). Individuals were explicitly encouraged to explore personalized strategies. Participants’ self-reported strategies fell into three broad categories: motivational thoughts (e.g., coach cheering them on), goal achievement/orientation (e.g., qualifying for a race), or vivid imagery (e.g., rescuing a loved one). Example reports include “Running down a line where thousands of people were giving me high-fives” and “I can do anything I put my mind to.” Each trial lasted 20 s. Trial order was randomized with each trial type presented five times, separated by a variable inter-trial interval of 3.5–5.5 s.

Participants next performed three Training runs. ACTIVATE trials contained rt-fMRI neurofeedback (Voyvodic, 1999; for reviews, see deCharms, 2007; Weiskopf et al., 2004) indicating the current average level of VTA or NAcc BOLD activation (VTA Feedback and NAcc Feedback groups, respectively), updated approximately one per second. Training runs consisted of three trial types: COUNT, ACTIVATE, and REST. COUNT and ACTIVATE instructions were identical to Pre-test. New instructional cues were presented to avoid transfer of conditioned responses to the cues (Figure 1). REST trials controlled for the fluctuating thermometer shown during ACTIVATE trials.

During ACTIVATE trials, a thermometer indicating the current level of VTA or NAcc BOLD activation (detailed in Feedback Stimulus) was presented. Participants were instructed to keep the thermometer level as high as possible and to update strategies as needed. During REST trials, participants were presented with a thermometer and told that the height was randomly determined and did not reflect ongoing brain activity. They were instructed to relax and not think of anything in particular. Trial order was pseudorandomly determined such that the first trial—and every third trial thereafter—was a COUNT trial for purposes of baseline calculations. Each trial type was presented five times, separated by a variable inter-trial interval of 3.5–5.5 s.

The MRI session ended with a Post-test identical in structure to the Pre-test. During Post-test ACTIVATE trials, participants were encouraged to apply their most effective strategies.

VC and FF Groups

Separate groups performed control versions of the task to determine whether participants can increase BOLD activation in the absence of rt-fMRI neurofeedback (VC group, n = 20) or when provided with noise, false neurofeedback (FF group, n = 14). Tasks for the FF group were identical to the VTA and NAcc Feedback groups, with the exception that the thermometer (during both ACTIVATE and REST trials) displayed a random pattern of values, drawn from a zero-centered normal distribution, unrelated to BOLD signal. Participants in the FF group were given the same instructions for the Test and Training runs as the veridical feedback groups and therefore were given the impression they would receive veridical neurofeedback.

Tasks for the VC group were identical to the feedback groups, with the exception that the feedback thermometer (during ACTIVATE and REST trials) displayed a predictable pattern of values unrelated to BOLD signal. Participants in the VC group were given the same instructions for the Pre-test and Post-test as all other groups. During Training, they were informed the purpose of the study was to test the effects of visual stimulation on self-generated motivational states. They were told to try to increase motivation during the presence of a predictable visual stimulus. Participants were shown the stimulus prior to scanning to be familiarized with the pattern of movement. Following scan completion, participants were asked whether they believed the pattern reflected anything other than a predetermined pattern. All participants were debriefed following the scanning session.

Feedback Stimulus

For the VTA and NAcc Feedback groups, rt-fMRI neurofeedback was provided via a thermometer during Training. The height of the thermometer represented the current level of VTA or NAcc BOLD activation. VTA neurofeedback was calculated using a weighted average from the probabilistic VTA atlas (see Supplemental Experimental Procedures) and converted to percent signal change. NAcc neurofeedback was calculated using a non-weighted average from the NAcc ROI and converted to percent signal change, consistent with Greer et al. (2014). For both groups, positive thermometer values were displayed in red; negative values were displayed in blue (REST trial displays were yellow and green). In addition, the numerical value was presented below the thermometer. Percent signal change was calculated against a trial-specific baseline defined as the average raw VTA or NAcc signal across the last three seconds of the previous COUNT trial. Using COUNT trials for this purpose enabled baseline measures to be calculated during a standardized behavior. The thermometer bar height was updated as soon as each new time point was acquired (approximately one per second). To ensure similar graphical feedback across participants, the thermometer range updated dynamically according to the variability in each participant’s fMRI signal (for groups that received valid feedback). A value of 0.5% was displayed as a solid line across the thermometer, providing a goal for participants. Following each ACTIVATE trial, a dashed line appeared on the thermometer indicating the average performance throughout the last trial. This line reappeared on the next ACTIVATE trial as a reference. This benchmark was given to all groups except the VC group (it was not relevant). Individuals in the VTA Feedback, NAcc Feedback, and FF groups were informed about the hemodynamic delay and the inherent noisiness of the BOLD signal and were encouraged to stick with one strategy for the duration of the trial. If they felt they were not successful in activating the target ROI, participants were encouraged to switch strategies on the next trial.

Offline Standard fMRI Analysis

Preprocessing

FMRI analysis was performed using FSL (http://www.fmrib.ox.ac.uk/fsl), v.5.0.1. Data processing was conducted using FEAT. The following pre-processing steps were applied: motion correction using MCFLIRT (Jenkinson et al., 2002), slice-timing correction using Fourier-space time series phase-shifting, non-brain removal using BET (Smith, 2002), spatial smoothing using a FWHM 4.0-mm Gaussian kernel, grand-mean intensity normalization of the entire 4D dataset by a single multiplicative factor, and high pass temporal filtering (Gaussian-weighted least-squares straight line fitting, with sigma = 40.0 s). All images were co-registered to subject-specific T1-weighted anatomical images and normalized to MNI space. Physiological noise was removed using FSL’s Physiological Noise Modeling toolbox (Brooks et al., 2008). Model parameters were chosen to optimize brainstem signal (Harvey et al., 2008) and included third-order cardiac terms, fourth-order respiration terms, and first-order interaction terms. Cardiac and respiration rates and respiration volume per time (RVT) were included as confounds regressors. For ROI analyses, the raw data were run through a preprocessing model containing only confound regressors (e.g., motion, physiological signals). The residuals of this model represented denoised BOLD signal and were used to extract ROI time series from individual runs. Whole-volume analyses and functional connectivity analyses employed models in which confound and task-related regressors were combined in the same design matrix.

ROI Analyses

We extracted a weighted-average time series of denoised BOLD signal from the VTA from each run. To examine sustained VTA activation, ERAs were calculated for the ACTIVATE > COUNT contrast. For each trial, activation at each time point was calculated relative to a pre-trial baseline (mean signal over 3-s window immediately preceding trial onset), allowing for examination of the response profile by condition. Within a run, trial time courses were first averaged by condition and then subtracted to produce ACTIVATE > COUNT comparisons. Similar ERAs were also calculated for the ACTIVATE > REST contrast during Training runs. A similar procedure was done for the NAcc ROI (non-weighted time courses).

Functional Connectivity Analysis

We asked how correlated activity across each run between pairs of ROIs within the mesolimbic network was affected by the source of neurofeedback signal (VTA or NAcc). We focused on six ROIs: the VTA, NAcc, left HPC (LHPC), right HPC (RHPC), left caudate nucleus (LCAUD), and right caudate nucleus (RCAUD) due to known anatomical connections between these regions (Gasbarri et al., 1997; Haber and Knutson, 2010; Lisman and Grace, 2005; Phillipson and Griffiths, 1985). We examined functional correlations between the following nine ROI pairs: VTA-NAcc, VTA-LHPC, VTA-RHPC, VTA-LCAUD, VTA-RCAUD, NAcc-LHPC, NAcc-RHPC, NAcc-LCAUD, and NAcc-RCAUD. Changes in functional connectivity strength across groups (Z-scored Pearson’s r values) were assessed using a 2 (run: Pre-Test, Post-Test) × 2 (group: VTA Feedback, NAcc Feedback) repeated-measures ANOVA.

Whole-Volume Results

Whole-volume results are reported in the Supplemental Information for the primary contrast of interest, ACTIVATE versus COUNT.

Statistical Notes

All post hoc t tests are two-tailed. Huynh-Feldt corrections were utilized to correct for sphericity violations, and the sequential Bonferroni technique was used to correct for multiple comparisons when appropriate (Holm, 1979; Rice, 1989).

Supplementary Material

Supplemental Information

Highlights.

  • VTA self-activation was inconsistent in untrained participants

  • After neurofeedback training, participants self-activated VTA without external cues

  • Control groups and NAcc feedback group failed to self-activate VTA or NAcc

  • After VTA neurofeedback training, connectivity in mesolimbic networks increased

In Brief.

Using neurofeedback, MacInnes and Dickerson et al. show that healthy individuals learn to sustain activation in the dopaminergic midbrain during and after training. The demonstration that humans can volitionally activate neuromodulatory regions holds promise for new educational and clinical interventions.

Acknowledgments

The authors thank C. Petty and J. Voyvodic for valuable technical support; D. Paulsen, R.M. Carter, E. Sumner, C. Coutlee, S. Floresco, S. Kollins, S. Madlon-Kay, M. Scult, T. Strauman, V. Murty, L. Davachi, S. Hakimi, and K. Dzirasa for comments and suggestions; and staff at the Brain Imaging and Analysis Center for MR imaging support. Supported by a NIMH BRAINS award (MH9743) to R.A.A. and a NIMH F32 (MH100764) to K.C.D. Additional support is provided by the Alfred P. Sloan Foundation, the Esther A. & Joseph Klingenstein Fund, and the Dana Foundation.

Footnotes

SUPPLEMENTAL INFORMATION

Supplemental Information includes Supplemental Experimental Procedures, four figures, and three tables and can be found with this article online at http://dx.doi.org/10.1016/j.neuron.2016.02.002.

AUTHOR CONTRIBUTIONS

Conceptualization, J.J.M., K.C.D., N.C., and R.A.A.; Methodology, J.J.M., K.C.D., N.C., and R.A.A.; Software, J.J.M. and K.C.D.; Formal Analysis, J.J.M., K.C.D., and R.A.A.; Investigation, J.J.M. and K.C.D.; Resources, R.A.A.; Writing – Original Draft, J.J.M., K.C.D., and R.A.A.; Writing – Review and Editing, J.J.M., K.C.D., N.C., and R.A.A.; Visualization, J.J.M.; Supervision, R.A.A.; Project administration, R.A.A.; Funding acquisition, K.C.D., N.C., and R.A.A.

REFERENCES

  1. Adcock RA, Thangavel A, Whitfield-Gabrieli S, Knutson B, Gabrieli JD. Reward-motivated learning: mesolimbic activation precedes memory formation. Neuron. 2006;50:507–517. doi: 10.1016/j.neuron.2006.03.036. [DOI] [PubMed] [Google Scholar]
  2. Alcaro A, Huber R, Panksepp J. Behavioral functions of the mesolimbic dopaminergic system: an affective neuroethological perspective. Brain Res. Brain Res. Rev. 2007;56:283–321. doi: 10.1016/j.brainresrev.2007.07.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Ballard IC, Murty VP, Carter RM, MacInnes JJ, Huettel SA, Adcock RA. Dorsolateral prefrontal cortex drives mesolimbic dopaminergic regions to initiate motivated behavior. J. Neurosci. 2011;31:10340–10346. doi: 10.1523/JNEUROSCI.0895-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Beeler JA, Daw N, Frazier CRM, Zhuang X. Tonic dopamine modulates exploitation of reward learning. Front. Behav. Neurosci. 2010;4:170. doi: 10.3389/fnbeh.2010.00170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Beierholm U, Guitart-Masip M, Economides M, Chowdhury R, Düzel E, Dolan R, Dayan P. Dopamine modulates reward-related vigor. Neuropsychopharmacology. 2013;38:1495–1503. doi: 10.1038/npp.2013.48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Botvinick MM, Huffstetler S, McGuire JT. Effort discounting in human nucleus accumbens. Cogn. Affect. Behav. Neurosci. 2009;9:16–27. doi: 10.3758/CABN.9.1.16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Braver TS, Barch DM. A theory of cognitive control, aging cognition, and neuromodulation. Neurosci. Biobehav. Rev. 2002;26:809–817. doi: 10.1016/s0149-7634(02)00067-2. [DOI] [PubMed] [Google Scholar]
  8. Braver TS, Krug MK, Chiew KS, Kool W, Westbrook JA, Clement NJ, Adcock RA, Barch DM, Botvinick MM, Carver CS, et al. MOMCAI group. Mechanisms of motivation-cognition interaction: challenges and opportunities. Cogn. Affect. Behav. Neurosci. 2014;14:443–472. doi: 10.3758/s13415-014-0300-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bromberg-Martin ES, Matsumoto M, Hikosaka O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron. 2010;68:815–834. doi: 10.1016/j.neuron.2010.11.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Brooks JCW, Beckmann CF, Miller KL, Wise RG, Porro CA, Tracey I, Jenkinson M. Physiological noise modelling for spinal functional magnetic resonance imaging studies. Neuroimage. 2008;39:680–692. doi: 10.1016/j.neuroimage.2007.09.018. [DOI] [PubMed] [Google Scholar]
  11. Callan DE, Schweighofer N. Positive and negative modulation of word learning by reward anticipation. Hum. Brain Mapp. 2008;29:237–249. doi: 10.1002/hbm.20383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cools R, D’Esposito M. Inverted-U-shaped dopamine actions on human working memory and cognitive control. Biol. Psychiatry. 2011;69:e113–e125. doi: 10.1016/j.biopsych.2011.03.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. D’Ardenne K, McClure SM, Nystrom LE, Cohen JD. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science. 2008;319:1264–1267. doi: 10.1126/science.1150605. [DOI] [PubMed] [Google Scholar]
  14. Dayan P, Niv Y. Reinforcement learning: the good, the bad and the ugly. Curr. Opin. Neurobiol. 2008;18:185–196. doi: 10.1016/j.conb.2008.08.003. [DOI] [PubMed] [Google Scholar]
  15. deCharms RC. Reading and controlling human brain activation using real-time functional magnetic resonance imaging. Trends Cogn. Sci. 2007;11:473–481. doi: 10.1016/j.tics.2007.08.014. [DOI] [PubMed] [Google Scholar]
  16. Duncan K, Tompary A, Davachi L. Associative encoding and retrieval are predicted by functional connectivity in distinct hippocampal area CA1 pathways. J. Neurosci. 2014;34:11188–11198. doi: 10.1523/JNEUROSCI.0521-14.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003;299:1898–1902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]
  18. Gasbarri A, Sulli A, Packard MG. The dopaminergic mesencephalic projections to the hippocampal formation in the rat. Prog. Neuropsychopharmacol. Biol. Psychiatry. 1997;21:1–22. doi: 10.1016/s0278-5846(96)00157-1. [DOI] [PubMed] [Google Scholar]
  19. Goldman-Rakic PS. The cortical dopamine system: Role in memory and cognition. In: Goldstein DS, Eisenhofer G, McCarty R, editors. In Advances in Pharmacology. Elsevier; 1997. pp. 707–711. [DOI] [PubMed] [Google Scholar]
  20. Goto Y, Otani S, Grace AA. The Yin and Yang of dopamine release: a new perspective. Neuropharmacology. 2007;53:583–587. doi: 10.1016/j.neuropharm.2007.07.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Greer SM, Trujillo AJ, Glover GH, Knutson B. Control of nucleus accumbens activity with neurofeedback. Neuroimage. 2014;96:237–244. doi: 10.1016/j.neuroimage.2014.03.073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Haber SN, Knutson B. The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology. 2010;35:4–26. doi: 10.1038/npp.2009.129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hall J, Parkinson JA, Connor TM, Dickinson A, Everitt BJ. Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviour. Eur. J. Neurosci. 2001;13:1984–1992. doi: 10.1046/j.0953-816x.2001.01577.x. [DOI] [PubMed] [Google Scholar]
  24. Harvey AK, Pattinson KTS, Brooks JCW, Mayhew SD, Jenkinson M, Wise RG. Brainstem functional magnetic resonance imaging: disentangling signal from physiological noise. J. Magn. Reson. Imaging. 2008;28:1337–1344. doi: 10.1002/jmri.21623. [DOI] [PubMed] [Google Scholar]
  25. Holm S. A Simple Sequentially Rejective Multiple Test Procedure. Scand. J. Stat. 1979;6:65–70. [Google Scholar]
  26. Howe MW, Tierney PL, Sandberg SG, Phillips PE, Graybiel AM. Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature. 2013;500:575–579. doi: 10.1038/nature12475. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Huettel SA, Song AW, McCarthy G. Functional Magnetic Resonance Imaging. Second. Sinauer; 2009. [Google Scholar]
  28. Jahanshahi M. Willed action and its impairments. Cogn. Neuropsychol. 1998;15:483–533. doi: 10.1080/026432998381005. [DOI] [PubMed] [Google Scholar]
  29. James W. The Principles of Psychology. Vol. 2. Dover Publications; 1950. [Google Scholar]
  30. Jenkinson M, Bannister P, Brady M, Smith S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage. 2002;17:825–841. doi: 10.1016/s1053-8119(02)91132-8. [DOI] [PubMed] [Google Scholar]
  31. Lee W, Reeve J. Self-determined, but not non-self-determined, motivation predicts activations in the anterior insular cortex: an fMRI study of personal agency. Soc. Cogn. Affect. Neurosci. 2013;8:538–545. doi: 10.1093/scan/nss029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Lee W, Reeve J, Xue Y, Xiong J. Neural differences between intrinsic reasons for doing versus extrinsic reasons for doing: an fMRI study. Neurosci. Res. 2012;73:68–72. doi: 10.1016/j.neures.2012.02.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Leotti LA, Delgado MR. The inherent reward of choice. Psychol. Sci. 2011;22:1310–1318. doi: 10.1177/0956797611417005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Lisman JE, Grace AA. The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron. 2005;46:703–713. doi: 10.1016/j.neuron.2005.05.002. [DOI] [PubMed] [Google Scholar]
  35. Lisman J, Grace AA, Düzel E. A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends Neurosci. 2011;34:536–547. doi: 10.1016/j.tins.2011.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Lou HC, Skewes JC, Thomsen KR, Overgaard M, Lau HC, Mouridsen K, Roepstorff A. Dopaminergic stimulation enhances confidence and accuracy in seeing rapidly presented words. J. Vis. 2011;11:15. doi: 10.1167/11.2.15. [DOI] [PubMed] [Google Scholar]
  37. Maia TV, Frank MJ. From reinforcement learning models to psychiatric and neurological disorders. Nat. Neurosci. 2011;14:154–162. doi: 10.1038/nn.2723. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Marder E. Neuromodulation of neuronal circuits: back to the future. Neuron. 2012;76:1–11. doi: 10.1016/j.neuron.2012.09.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Montague PR, Hyman SE, Cohen JD. Computational roles for dopamine in behavioural control. Nature. 2004;431:760–767. doi: 10.1038/nature03015. [DOI] [PubMed] [Google Scholar]
  40. Murayama K, Matsumoto M, Izuma K, Matsumoto K. Neural basis of the undermining effect of monetary reward on intrinsic motivation. Proc. Natl. Acad. Sci. USA. 2010;107:20911–20916. doi: 10.1073/pnas.1013305107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Murty VP, Shermohammed M, Smith DV, Carter RM, Huettel SA, Adcock RA. Resting state networks distinguish human ventral tegmental area from substantia nigra. Neuroimage. 2014;100:580–589. doi: 10.1016/j.neuroimage.2014.06.047. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Nishino H, Ono T, Muramoto K, Fukuda M, Sasaki K. Neuronal activity in the ventral tegmental area (VTA) during motivated bar press feeding in the monkey. Brain Res. 1987;413:302–313. doi: 10.1016/0006-8993(87)91021-3. [DOI] [PubMed] [Google Scholar]
  43. Niv Y, Duff MO, Dayan P. Dopamine, uncertainty and TD learning. Behav. Brain Funct. 2005;1:6. doi: 10.1186/1744-9081-1-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Niv Y, Daw ND, Joel D, Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 2007;191:507–520. doi: 10.1007/s00213-006-0502-4. [DOI] [PubMed] [Google Scholar]
  45. Pessoa L, Engelmann JB. Embedding reward signals into perception and cognition. Front. Neurosci. 2010;4:1–8. doi: 10.3389/fnins.2010.00017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Phillipson OT, Griffiths AC. The topographic order of inputs to nucleus accumbens in the rat. Neuroscience. 1985;16:275–296. doi: 10.1016/0306-4522(85)90002-8. [DOI] [PubMed] [Google Scholar]
  47. Plato . The Republic of Plato. Basic Books; 1991. [Google Scholar]
  48. Rice WR. Analyzing Tables of Statistical Tests. Evolution. 1989;43:223. doi: 10.1111/j.1558-5646.1989.tb04220.x. [DOI] [PubMed] [Google Scholar]
  49. Roitman MF, Stuber GD, Phillips PEM, Wightman RM, Carelli RM. Dopamine operates as a subsecond modulator of food seeking. J. Neurosci. 2004;24:1265–1271. doi: 10.1523/JNEUROSCI.3823-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Salamone JD, Correa M. The mysterious motivational functions of mesolimbic dopamine. Neuron. 2012;76:470–485. doi: 10.1016/j.neuron.2012.10.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Schott BH, Minuzzi L, Krebs RM, Elmenhorst D, Lang M, Winz OH, Seidenbecher CI, Coenen HH, Heinze HJ, Zilles K, et al. Mesolimbic functional magnetic resonance imaging activations during reward anticipation correlate with reward-related ventral striatal dopamine release. J. Neurosci. 2008;28:14311–14319. doi: 10.1523/JNEUROSCI.2058-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Schultz W. Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 2007;30:259–288. doi: 10.1146/annurev.neuro.28.061604.135722. [DOI] [PubMed] [Google Scholar]
  53. Shibata K, Watanabe T, Sasaki Y, Kawato M. Perceptual learning incepted by decoded fMRI neurofeedback without stimulus presentation. Science. 2011;334:1413–1415. doi: 10.1126/science.1212003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Shohamy D, Adcock RA. Dopamine and adaptive memory. Trends Cogn. Sci. 2010;14:464–472. doi: 10.1016/j.tics.2010.08.002. [DOI] [PubMed] [Google Scholar]
  55. Smith SM. Fast robust automated brain extraction. Hum. Brain Mapp. 2002;17:143–155. doi: 10.1002/hbm.10062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Stuber GD, Roitman MF, Phillips PEM, Carelli RM, and Wightman RM. Rapid dopamine signaling in the nucleus accumbens during contingent and noncontingent cocaine administration. Neuropsychopharmacology. 2005;30:853–863. doi: 10.1038/sj.npp.1300619. [DOI] [PubMed] [Google Scholar]
  57. Sulzer J, Sitaram R, Blefari ML, Kollias S, Birbaumer N, Stephan KE, Luft A, Gassert R. Neurofeedback-mediated self-regulation of the dopaminergic midbrain. Neuroimage. 2013;83:817–825. doi: 10.1016/j.neuroimage.2013.05.115. [DOI] [PubMed] [Google Scholar]
  58. Takahashi YK, Roesch MR, Wilson RC, Toreson K, O’Donnell P, Niv Y, Schoenbaum G. Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 2011;14:1590–1597. doi: 10.1038/nn.2957. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Totah NKB, Kim Y, Moghaddam B. Distinct prestimulus and poststimulus activation of VTA neurons correlates with stimulus detection. J. Neurophysiol. 2013;110:75–85. doi: 10.1152/jn.00784.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Treadway MT, Buckholtz JW, Cowan RL, Woodward ND, Li R, Ansari MS, Baldwin RM, Schwartzman AN, Kessler RM, Zald DH. Dopaminergic mechanisms of individual differences in human effort-based decision-making. J. Neurosci. 2012;32:6170–6176. doi: 10.1523/JNEUROSCI.6459-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Tricomi EM, Delgado MR, Fiez JA. Modulation of caudate activity by action contingency. Neuron. 2004;41:281–292. doi: 10.1016/s0896-6273(03)00848-1. [DOI] [PubMed] [Google Scholar]
  62. Volkow ND, Wang G-J, Kollins SH, Wigal TL, Newcorn JH, Telang F, Fowler JS, Zhu W, Logan J, Ma Y, et al. Evaluating dopamine reward pathway in ADHD: clinical implications. JAMA. 2009;302:1084–1091. doi: 10.1001/jama.2009.1308. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Volkow ND, Wang G-J, Newcorn JH, Kollins SH, Wigal TL, Telang F, Fowler JS, Goldstein RZ, Klein N, Logan J, et al. Motivation deficit in ADHD is associated with dysfunction of the dopamine reward pathway. Mol. Psychiatry. 2011;16:1147–1154. doi: 10.1038/mp.2010.97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Voyvodic JT. Real-time fMRI paradigm control, physiology, and behavior combined with near real-time statistical analysis. Neuroimage. 1999;10:91–106. doi: 10.1006/nimg.1999.0457. [DOI] [PubMed] [Google Scholar]
  65. Weiskopf N, Scharnowski F, Veit R, Goebel R, Birbaumer N, Mathiak K. Self-regulation of local brain activity using real-time functional magnetic resonance imaging (fMRI) J. Physiol. Paris. 2004;98:357–373. doi: 10.1016/j.jphysparis.2005.09.019. [DOI] [PubMed] [Google Scholar]
  66. Wimmer GE, Daw ND, Shohamy D. Generalization of value in reinforcement learning by humans. Eur. J. Neurosci. 2012;35:1092–1104. doi: 10.1111/j.1460-9568.2012.08017.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Wittmann BC, Schott BH, Guderian S, Frey JU, Heinze H-J, Düzel E. Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation. Neuron. 2005;45:459–467. doi: 10.1016/j.neuron.2005.01.010. [DOI] [PubMed] [Google Scholar]
  68. Wolosin SM, Zeithamova D, Preston AR. Reward modulation of hippocampal subfield activation during successful associative encoding and retrieval. J. Cogn. Neurosci. 2012;24:1532–1547. doi: 10.1162/jocn_a_00237. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Information
NIHMS810940-supplement-Supplemental_Information.pdf (10.9MB, pdf)

RESOURCES