Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2009 Jun 10;29(23):7591–7598. doi: 10.1523/JNEUROSCI.5335-08.2009

Nuclei Accumbens Phase Synchrony Predicts Decision-Making Reversals Following Negative Feedback

Michael X Cohen 1,3,✉, Nikolai Axmacher 1, Doris Lenartz 4, Christian E Elger 1, Volker Sturm 4, Thomas E Schlaepfer 2,5
PMCID: PMC2725399  NIHMSID: NIHMS124497  PMID: 19515927

Abstract

The nucleus accumbens plays a key role in reinforcement-guided behaviors. Here, we report that electrophysiological oscillatory phase synchrony between the two nuclei accumbens may play a crucial role in using negative feedback to guide decision making. We recorded local field potentials from the human nucleus accumbens and the medial frontal cortex (via surface EEG) from patients who had deep brain stimulation electrodes implanted. Patients performed a reversal learning task in which they decided whether to alter their decision strategy following monetary losses. Strategy switches following losses were preceded by enhanced theta (4–8 Hz) phase synchrony between the nuclei accumbens, and a break-down of gamma (20–80 Hz)–α (8–12 Hz) coupling. Furthermore, the strength of the intersite phase synchrony predicted response time adjustments in the subsequent trial. These findings suggest that a neural network including the nucleus accumbens bilaterally becomes functionally connected via theta phase synchrony to signal the need to adjust behavior.

Introduction

Although relatively small in size, the nucleus accumbens plays a major role in both normal and abnormal reward processing, reinforcement learning, and impulsivity (Deadwyler et al., 2004). Its precise function remains debated; it has variously been ascribed to act as a “gateway” between motivation and action (Groenewegen et al., 1996), code for reward value (Knutson et al., 2001), signal reward prediction error (McClure et al., 2003; Yacubian et al., 2006), and use reinforcements to adapt goal-directed behavior (Cools et al., 2002; Schoenbaum and Setlow, 2003; Pothuizen et al., 2005). Furthermore, little is known about the electrophysiological mechanisms by which these functions may be realized in humans, and by which the nucleus accumbens communicates with other brain structures. Here, we used a unique opportunity to advance our understanding of nucleus accumbens functioning by recording electrophysiological potentials in vivo in humans.

The nucleus accumbens is part of a network of anatomically and functionally connected regions, including midbrain dopamine structures and the medial frontal cortex (Mogenson et al., 1983; Haber et al., 2006), that work together to adapt goal-directed behavior. How do these structures communicate with one another to form functionally connected networks? Neurophysiological oscillations might reflect a mechanism by which neural networks become functionally connected, and activities (e.g., spike timing) at both distal and local spatial scales are coordinated (Fries, 2005; Kasanetz et al., 2006; Tort et al., 2007). Oscillatory activities in the medial frontal cortex and nucleus accumbens have been linked to reward processing and behavior adaptation (Cohen et al., 2007, 2008b; Marco-Pallares et al., 2008; Münte et al., 2008), and synchronous oscillations within subcortical and cortical networks have been linked to learning and decision making (Paz et al., 2006; Pesaran et al., 2008). Based on the hypothesis that synchronous oscillations reflect coordinated activity of distal neural networks, we expected that the strength of oscillatory synchrony among regions within this network (e.g., between the left and right nucleus accumbens, and between the nuclei accumbens and medial frontal cortex), increases when the network signals the need to adjust behavior based on reinforcements.

In addition to coordinating the activity of spatially distal neural networks, oscillations may also coordinate the activity of local neural networks (i.e., what can be recorded from within one electrode), for example, if slow oscillations regulate the timing of faster oscillations or action potentials. Such “cross-frequency coupling” is thought to reflect a gating mechanism by which local processes (e.g., action potentials or bursts of high frequency activity) are regulated by the present neural state (e.g., oscillation phase). Cross-frequency coupling, particularly gamma-theta and gamma-α, has been observed in several cortical and subcortical regions (Chrobak and Buzsáki, 1998; Lakatos et al., 2005; Mormann et al., 2005; Canolty et al., 2006). In the human nucleus accumbens, bursts of gamma (∼40–80 Hz) are synchronized with the phase of α (8–12) during reward learning tasks (Cohen et al., 2008b). Here, we examined whether cross-frequency coupling in the nucleus accumbens is modulated by how reinforcements are used to adapt decision making.

Materials and Methods

Patients.

All six patients (three male; aged 45–64, average: 52) suffered from treatment-refractory (refractory to multiple medications, psychotherapy, and electroconvulsive therapy) major depression, and qualified for our clinical investigation into the effects of deep brain stimulation (DBS) for treatment. Electrode placement was planned using MRIs, as described previously (Sturm et al., 2003; Schlaepfer et al., 2008). The target structure was the posteroventromedial part of the nucleus accumbens (Fig. 1A). The location of electrode placement was made entirely on clinical grounds. This experiment, and the larger clinical study of the use of DBS as a treatment option for major depression, was approved by the ethics committees at the Universities of Bonn and Cologne. This study is registered with the Trials Registry (www.clinicaltrials.gov) under the number NCT00122031. More information regarding the potential clinical efficacy of DBS to the nucleus accumbens is presented by Schlaepfer et al. (2008).

Figure 1.

Figure 1.

Open in a new tab

A, Placement of DBS electrodes in nucleus accumbens in one patient. Displayed is an MRI from one patient used in electrode placement planning. The “X” (ventral end of the lines) shows the location of the electrode probe through this slice. B, Overview of experimental design and timing of events.

Task.

To examine the role of the nucleus accumbens in feedback-guided learning, we used a probabilistic reversal learning task (Fig. 1B). In each of 200 trials, two sides of a Euro 1 cent coin were displayed on the left and right sides of the screen (Fig. 1B). Patients selected one of the two, using the left or right mouse buttons in the laptop's mouse, using their left or right-hand. The stimuli remained onscreen until the patient pressed a button, at which point a green square highlighted the selected coin and “+0.10 €” (reward) or “–0.10 €” (loss) appeared between the stimuli for 1000 ms. At the start of the experiment, the rule (i.e., which stimulus rewarded 75% of the time) was chosen randomly. On each selection of the correct stimulus, patients had a 0.75 probability of winning, and on each selection of the incorrect stimulus, patients had a 0.25 probability of winning. Probabilities on each trial were independent. The rule reversed after a minimum of 12 trials and at least 71% accuracy over a sliding window of the previous 7 trials. A randomly jittered intertrial-interval between 2 and 4 s separated each trial. Before the experiment, there was a training session in which patients were given verbal instructions and practiced the task for at least two reversals and verbal confirmation that they understood the task.

EEG recording.

Intracranial EEG recordings were conducted in a quiet testing room the day after surgical implantation of the DBS electrodes. The DBS electrodes are Medtronic model 3387, and are made of a mixture of Platinum/Iridium (90/10%). At this time, electrode leads remained externalized and could be hooked up to our mobile EEG recording system. Continuous EEG data were sampled at 1000 Hz with a 300 Hz anti-alias filter, and referenced to linked mastoids. The recordings reported here were taken from the ventral-most contact in each hemisphere. Anatomically, the ventral-most contacts are located in the purported shell of the nucleus accumbens, as evidenced by post-implantation CT scans. Nucleus accumbens data were rereferenced to the most dorsal contact in each probe; this ensures that reported activity is generated in the ventral striatum. The results looked similar when referenced to the external mastoids, suggesting that the field potentials are locally generated regardless of the reference position. Data from the two contacts between the dorsal-most contact (used as the reference) and the ventral-most contact (data from which are reported here) looked similar in temporal characteristics, but smaller in amplitude. We collected surface EEG data from between 3 and 12 electrodes for each patient; all but one patient had electrode position at frontal cortex (Fz) (data from that patient were excluded from Fz-related analyses). Surface EEG data were referenced to linked mastoids. Patients had taken their standard antidepressant medication, but were not sedated. Patients sat in a comfortable chair in front of a desk and performed the experiment on a laptop computer. The laptop was equipped with a parallel output cable that delivered triggers to the EEG recording system at the onset of each visual stimulus and button press with millisecond precision.

At the time of recording, patients did not receive accumbens stimulation. Although some immediate beneficial effects of DBS (i.e., within minutes) have been reported, clinical effects of DBS continue to develop after weeks to months of stimulation (Schlaepfer et al., 2008). Furthermore, DBS has a complicated and not fully understood pattern of effects in both up- and down-stream regions (McIntyre et al., 2004; McCracken and Grace, 2007). For these reasons, we did not perform our experiment both before and after stimulation.

Event-related potential analyses.

To compute event-related potentials (ERPs), we averaged the trial data for each condition (win, loss/stay, loss/switch) from −200 until +1200 ms surrounding feedback onset. Single trial EEG epochs were baseline-subtracted using a −200–0 ms window and averaged together. Finally, ERPs for each patient were filtered from 0 to 20 Hz before group averaging. Analyses were conducted using repeated-measures ANOVAs of voltage changes from averaged windows 0–1200 ms after feedback in 300 ms windows using SPSS 16.0 software package.

Oscillation power and phase synchrony analyses.

Time-frequency decomposition was conducted via wavelet analysis, in which the EEG time series of each trial was convolved with a set of complex Morlet wavelets, defined as a Gaussian-windowed complex sine wave: ei2πtƒ e−t2/(2*σ2). t is time, ƒ is frequency, which increased from 2.5 to 160 Hz in 60 logarithmically spaced steps. σ defines the width of each frequency band, and is set according to 4.5/(2πƒ), which provides an adequate trade-off between time and frequency resolution. After convolution of the wavelet with the EEG, power is defined as the modulus of the resulting complex signal Z(t) (power time series: p(t) = real[z(t)]2 + imag[z(t)]2), and phase angle is defined as arctan(imag[z(t)]/real[z(t)]). The baseline was defined as average frequency power from –300 to –100 ms before the onset of the cue. This baseline was chosen to exclude any possible smearing from early low-frequency effects in the baseline period. Finally, stimulus-induced power time courses were normalized by converting the baseline-corrected signal to a decibel (dB) scale (10*log10[power/baseline]); this allows a direct comparison of effects across frequency bands and patients. We also computed fast Fourier transforms of the field potentials during and immediately following the task. Whereas posttask activity was characterized by a 1/frequency-like shape, the task elicited oscillations in the low theta/α ranges (see supplemental Information, available at www.jneurosci.org as supplemental material).

Inter-site phase synchrony measures the extent to which oscillation phase angle differences between electrodes are consistent over trials at each time/frequency point (if the oscillations are synchronized, their phase angle differences will be constant):

graphic file with name zns02309-6344-m01.jpg

where n is the number of points, and ϕj and ϕk are the phase angles of electrodes j and k. Inter-trial phase coherence measures the extent to which oscillation phase angles take on similar values at each time/frequency point across trials (Lachaux et al., 1999), and is defined as

graphic file with name zns02309-6344-m02.jpg

To compute statistics, we used repeated-measures ANOVAs in SPSS v16. To extract the data used in ANOVAs, we first identified the time-frequency peak within a window of 0–600 ms and 3–12 Hz for each patient, averaged across conditions. Then, we took the average time-frequency data in a window of 200 ms and 3 Hz surrounding each patient's time-frequency peak, separate for each condition. These values were entered into repeated-measures ANOVAs. Huynh–Feldt corrections were applied when appropriate, and adjusted degrees of freedom are reported in these cases. To assess how trial-by-trial phase synchrony predicted response time adjustments on subsequent trials, we calculated intersite phase synchrony as indicated above, but over time within each trial, instead of over trials at each time point. We used a time-frequency window of +200 to +600 ms, 5–7 Hz. This window corresponds to the maximum intersite phase synchrony time-frequency peak when averaging across all conditions. This procedure produced one phase synchrony value per trial, which was then subsequently correlated with response time differences between the subsequent and present trial (thus, positive values mean that patients were slower on the subsequent trial compared with the current trial). Synchrony values were Fisher-Z transformed to account for limited range. This was done separately for each condition within each patient. Correlation coefficients across patients were then submitted to a one-sample t test against the null hypothesis of an average coefficient of zero across patients.

Phase-amplitude coupling analyses.

To assess phase-amplitude coupling, we first bandpass filtered the data from each trial in the α range (8–12 Hz). Next, we identified all troughs of these waveforms from +100 to +900 ms following each experimental event. This time window was chosen because it includes all α cycles during feedback presentation. The raw signals were again convolved with complex wavelets as described above. After convolution, the power time series of each frequency range was extracted, normalized (Z-transformed), time-locked to the trough of the α wave, and averaged together across trials separately for each condition, and then across patients. To assess statistical significance of cross-frequency coupling (Fig. 4, z-values on the right of each plot) in each frequency band, we used the following boot-strapping method. First, we calculated the “modulation index,” which is the magnitude of the vector that results from projecting gamma power onto α phase (Canolty et al., 2006; Cohen et al., 2008b):

graphic file with name zns02309-6344-m03.jpg

where n is the number of time points in the series, at is the instantaneous power at a specific frequency ƒ at time t, and ϕt is the α phase value at time t. Next, we randomly resorted the power time series while preserving the α phase, and calculated a new m value (mboot). This procedure was repeated 200 times, resulting in a distribution of mboot values (i.e., projection vector magnitudes of frequency power on α phase) that can be expected by chance, given the observed α phase time series. mboot and observed modulation values were transformed according to the arcsine of the square root of the value [arcsin√(m)] to ensure that values would approach a normal distribution (Winer, 1971). Next, we calculated a z-value of the observed modulation index by subtracting the mean mboot and diving by the SD of mboot. This z-value thus reflects the standardized distance away from the distribution of gamma-α coupling expected by chance. Values exceeding a p < 0.001, FDR-corrected for multiple comparisons across frequency bands (Genovese et al., 2002) were considered statistically significant. To test for differences in the strength of cross-frequency coupling across conditions, we tested whether the difference in z-values between conditions exceeded a z-threshold set according to p < 0.001, FDR-corrected.

Figure 4.

Figure 4.

Open in a new tab

Bursts of gamma power are synchronized with the phase of simultaneously recorded α. Time-frequency plots were time-locked to the trough of each α-filtered oscillation wave instead of to the onset of the stimulus. The black line below each plot shows the averaged α-filtered EEG waveform. The black lines to the right of each plot show statistical significance of gamma-α synchrony at each frequency band, obtained through data-based boot strapping. The dotted gray line marks statistical significance at p < 0.001, corrected for multiple comparisons across frequency bands.

Results

Task overview and behavioral performance

Probabilistic reversal learning tasks are useful for investigating ventral striatal and prefrontal involvement in adaptive decision making in humans. We distinguished three types of trials: (1) win: trials in which patients receive positive feedback; (2) loss/stay: trials in which patients received negative feedback and selected the same coin in the subsequent trial as on the current trial; (3) loss/switch: trials in which patients received negative feedback and selected the opposite coin in the subsequent trial. Importantly, loss/stay and loss/switch trials do not differ on any characteristic except patients' decision whether to use that loss feedback to change their response strategy on the following trial. (Win trials could not be separated because win/switch trials are rare.)

Behaviorally, patients were able to adapt their decision-making strategies according to changes in response-reward contingencies. Patients selected the correct stimulus on 68% of trials (SE, 2.7%), which was significantly better than chance (t(5) = 7.08, p < 0.001). This is comparable with performance of young, healthy subjects in similar tasks we have previously conducted (67%) (Cohen et al., 2008a). Response times were 780, 803, and 872 (SE: 105, 84, and 135) ms for win, loss/stay, and loss/switch trials. These differences were not statistically significant (repeated-measures ANOVA: F(2,10) = 0.98, p = 0.412).

ERPs in the nucleus accumbens

We first examined the ERPs observed in the nucleus accumbens during each trial type. ERPs are the averaged feedback-locked time domain data, and provide insight into the timing of global activation of the nucleus accumbens. In general, feedback elicited a relatively slow-going potential that was maximally negative for loss/stay, followed by loss/switch and win feedback (Fig. 2A). The ERPs were significantly different between 400 and 800 ms (interaction of time × condition: F(3.5,17.8) = 3.57, p = 0.029), which was driven by a linearly increasingly negative potentials for win, loss/switch, and loss/stay conditions (linear term from ANOVA: F(1,5) = 12.69, p = 0.016). Loss/stay ERPs were numerically but not significantly more negative than loss/switch ERPs (F(1,5) = 4.98, p = 0.076). There were no main effects or interactions involving hemisphere (all F <1.67; p > 0.28). Notably, responses following wins were not significantly different from baseline in the left or right (t tests, all p > 0.086) nucleus accumbens at any time bin. These data indicate a general global activation of the nucleus accumbens by feedback stimuli that was enhanced following losses; we next examined the oscillatory dynamics of feedback-locked EEG.

Figure 2.

Figure 2.

Open in a new tab

Electrophysiological responses of the left (top row) and right (bottom row) human nucleus accumbens during reversal learning. A, Grand-averaged ERPs. B, Time-frequency plots show enhanced theta and gamma oscillatory responses to feedback (dB scale, relative to baseline), ∼300–600 ms following visual onset of reinforcement. Losses elicited larger theta power compared with wins; the difference between loss/stay and loss/switch was nonsignificant.

Oscillation power and intertrial phase coherence

We next examined whether frequency-band-specific activity in the nuclei accumbens reflected reward valence and/or a signal to adapt behavior. In general, feedback elicited oscillatory power increases in the theta (∼4–8 Hz) and gamma (∼20–80 Hz) ranges in the nuclei accumbens (Fig. 2B). Theta oscillation power was significantly different across the three conditions, which was driven by enhanced theta power for losses compared with rewards (F(1.7,8.9) = 4.6, p = 0.045), consistent with recordings of medial frontal activity (Cohen et al., 2007; Marco-Pallares et al., 2008). There were no significant differences between loss/stay and loss/switch trials in either hemisphere (paired-samples t(5) = 1.8 and –0.306, p = 0.131 and 0.772 for left and right accumbens), nor were there significant effects of or interactions with hemisphere (main effect and interaction term from ANOVA: F(1,5) = 1.86, p = 0.23; F(2,10) = 2.27, p = 0.15). Inter-trial phase coherence (see supplemental Information, available at www.jneurosci.org as supplemental material) exhibited a similar pattern of effects: Enhanced theta phase coherence for losses compared with wins (F(2,10) = 6.23, p = 0.018) and no effects of hemisphere (main effect and interaction term from ANOVA: F(1,5) = 0.62, p = 0.46; F(2,7.7) = 2.06, p = 0.19) or differences between loss/stay and loss/switch (paired-samples t(5) = 1.89, p = 0.12; t(5) = 0.848, p = 0.435 for left and right accumbens). Gamma power was significantly enhanced in all trials in both the left and right nucleus accumbens (t tests; all p < 0.034), but significant differences among conditions were not observed. Together, these data show task-related oscillatory activity in the theta and gamma ranges; theta activity was significantly enhanced following negative feedback.

Inter-site phase synchrony

In our next set of analyses, we used oscillatory phase synchrony as a millisecond-resolution index of the functional connectivity between the two nuclei accumbens, and between the nucleus accumbens and medial Fz. This analysis provides insight into how these spatially disparate regions might synchronize to form functional networks. Oscillatory phase synchrony in the theta band between the two nuclei accumbens was significantly greater than during the prestimulus baseline in all conditions (t(5) = 2.08, 2.75, 4.9; p = 0.046, 0.02, 0.002, for win, loss/stay, and loss/switch) (Fig. 3A). Synchrony was significantly greater for loss/switch compared with loss/stay (t(5) = 4.65, p = 0.006) (Fig. 3B), but was not significantly different between win and loss/stay (t(5) = 0.35, p = 0.37). Theta phase synchrony between Fz and left and right nucleus accumbens was significantly greater than baseline in all conditions (t tests, all p < 0.033) except for left accumbens-Fz synchrony during wins (t(4) = 1.78, p = 0.074). There were no significant differences among conditions for either left accumbens-Fz or right accumbens-Fz (ANOVA, all F <1.2, all p > 0.35) (Fig. 3D).

Figure 3.

Figure 3.

Open in a new tab

Inter-regional phase synchrony demonstrates that the nuclei accumbens were strongly phase synchronized during loss/switch trials, and the strength of this synchrony predicted subsequent response time adjustments. A shows interaccumbens phase synchrony for each condition; B shows the difference in phase synchrony between loss/switch and loss/stay (red colors indicate enhanced synchrony during loss/switch trials). C, The strength of interaccumbens phase synchrony on loss/switch trials predicted adjustments in response times on the subsequent trial. x-axis displays response time adjustment (subsequent–current trial; positive values mean the subsequent trial had a longer response time); y-axis displays Fisher-transformed phase synchrony values. Each dot is a single trial (unique colors/shapes for each patient), and lines correspond to correlation slope within each patient. D, Oscillatory phase synchrony between the nucleus accumbens and Fz was significantly greater compared with baseline but was not significantly different across conditions. Conditions are averaged together here and displayed separately in supplemental Figure S5, available at www.jneurosci.org as supplemental material.

To more closely link intersite phase synchrony to behavior, we correlated the strength of interaccumbens phase synchrony at 6 Hz (i.e., the center of the phase synchrony burst) on a trial-by-trial level with the extent to which patients adjusted their response times on the subsequent trial. We found that during loss/switch trials, stronger phase synchrony was associated with increased slowing on the subsequent trial (average correlation coefficient across patients: 0.18, p = 0.0017). Phase synchrony strength during loss/stay and win trials did not significantly predict subsequent response time adjustments (average correlation coefficients: 0.005 and 0.031, p = 0.687 and 0.238, respectively) (Fig. 3C). In other words, when patients received negative feedback and decided to switch their decision-making strategy, the strength of functional connectivity between the nuclei accumbens predicted the extent to which they slowed down on the following trial. Thus, the nuclei accumbens became significantly phase synchronized following feedback, and the strength of synchrony correlated with reaction time adjustments when patients switched their behavior.

Gamma power-α phase coupling

In our final set of analyses, we used cross-frequency coupling within a single electrode in the nucleus accumbens as an index of local functional organization. In these analyses, we time-locked the power spectrum to the trough of the simultaneous α (8–12 Hz)-filtered EEG, rather than to the onset of the feedback stimulus. Statistics were performed through boot-strapping techniques, in which the relation between the gamma power and α phase time series were repeatedly shuffled (see Materials and Methods). Consistent with our previous findings (Cohen et al., 2008b), bursts of gamma activity (∼20–80 Hz) were synchronized with the phase of α, occurring during α peaks (Fig. 4). The black line to the right of each plot in Figure 4 illustrates the probability of observing synchrony of that strength by chance, within each frequency band; the dotted line reflects a statistical cutoff of p < 0.001, correcting for multiple comparisons across frequencies (see Materials and Methods). These results show that high frequency oscillations from ∼20 to 100 Hz in the left and right nucleus accumbens were synchronized with α phase. Consistent with previous results (Cohen et al., 2008b), gamma power was not significantly coupled to theta (see supplemental Information, available at www.jneurosci.org as supplemental material). Notably, loss/switch trials were accompanied by a significant reduction in phase-amplitude coupling. In Figure 5,we illustrate the difference statistic between coupling during loss/switch and loss/stay trials at each frequency band (positive values indicate stronger coupling for loss/stay compared with loss/switch trials). The gray bars indicate a significant condition difference at p < 0.001. In the left nucleus accumbens, significant differences were observed in several frequency ranges, generally between ∼20 and ∼50 Hz. In the right nucleus accumbens, differences were significant in a higher frequency range, from ∼70 to 130 Hz.

Figure 5.

Figure 5.

Open in a new tab

Differences in gamma-α coupling between loss/stay and loss/switch trials. The x-axis represents frequency (in Hz), and the y-axis represents statistical difference values. The solid black line shows the difference in gamma-α coupling strength between loss/stay and loss/switch (positive values mean stronger coupling for loss/stay); gray regions indicate statistical significance at p < 0.001, corrected for multiple comparisons across frequency bands.

To establish a closer link between cross-frequency coupling and behavior adjustments, we correlated the gamma-α modulation on each trial with response time adjustments on the following trial. Although we did observe that the strength of gamma-α coupling predicted subsequent response time adjustments (and this was stronger during loss/stay compared with loss/switch trials) at a significance threshold of p < 0.05, this did not survive corrections for multiple comparisons across frequency bands (see supplemental Information, available at www.jneurosci.org as supplemental material). In our final analysis, we examined whether trial-by-trial fluctuations in cross-frequency coupling were related to intersite phase synchrony. As with the previous result, we observed that gamma-α coupling was correlated with intersite phase synchrony on a trial-by-trial level at an uncorrected significance threshold of p < 0.05, although these did not survive correction for multiple comparisons (see supplemental Information, available at www.jneurosci.org as supplemental material). In summary, feedback-related gamma-α synchrony diminished during loss/switch trials, and, to some extent, correlated with adjustments in subsequent reaction times.

Discussion

The nucleus accumbens is a core component of a neural network that uses reinforcements to adapt behavior. Using intracranial recordings from awake humans, we showed that oscillations in theta, α, and gamma frequency bands might in part underlie the functional organization of nucleus accumbens-linked networks during reversal learning. In particular, switches in decision-making strategy were preceded by enhanced oscillatory phase synchrony between the two nuclei accumbens, and a break-down of phase-amplitude coupling within each accumbens. These findings implicate both large- and small-scale nucleus accumbens-linked neural networks in signaling the need to adjust decision-making strategy according to negative feedback.

Reversal learning tasks are often used to investigate the role of the human ventral striatum and prefrontal cortex in reversal learning (Cools et al., 2002, 2006; Cohen et al., 2008a). In functional magnetic resonance imaging studies, ventral striatal hemodynamic responses are larger during loss compared with win feedback, and are relatively larger during loss/switch compared with loss/stay trials (Cools et al., 2002). Similarly, we found enhanced theta activity for losses compared with wins, consistent with results from medial frontal cortex (Cohen et al., 2007; Marco-Pallares et al., 2008), although differences between loss/stay and loss/switch trials were nonsignificant. These results suggest that one key function of the nucleus accumbens is to use negative feedback to signal the need to adjust behavior (Schoenbaum and Setlow, 2003; Cools et al., 2006). This may seem at-odds with other reports indicating that the nucleus accumbens codes for reward anticipation (Knutson et al., 2001) or reward prediction errors (McClure et al., 2003; Yacubian et al., 2006), which would predict maximal accumbens activity during rewards in the present task. In contrast, we observed no significant ERP or theta response to the rewarding feedback stimulus compared with baseline in the present task. It is likely that the nucleus accumbens' response to feedback valence depends on task demands, particularly how feedback may be used for guiding behavior.

Notably, our findings suggest that the nucleus accumbens' function may depend in part on its interactions with other brain regions: Inter-accumbens synchrony strengthened when patients received negative feedback and adjusted their decision making on the following trial. Phase synchrony may reflect the functional coupling among regions, or the extent to which two regions are transferring information or processing the same information (Varela et al., 2001; Fries, 2005; Steriade, 2006). Indeed, chemical or mechanical disruptions to the flow of information between the nucleus accumbens and orbitofrontal cortex (Block et al., 2007), or between the nucleus accumbens and the hippocampus (Goto and Grace, 2005), impair performance on tasks in which reinforcements must be used to guide decision making. Under the hypothesis that synchronized oscillations reflect inter-regional communication (Fries, 2005), during loss/switch trials the nuclei accumbens may have been “communicating” with each other to compute or coordinate the signal to adapt future behavior. This is further supported by the trial-by-trial correlations, in which relatively increased oscillatory phase synchrony between the two nuclei accumbens predicted the extent to which patients slowed down their responses on the following trial.

This interaccumbens synchrony effect could not be explained by an overall increase in oscillation power, because there were no significant differences in the theta range between loss/stay and loss/switch trials in either the left or right nucleus accumbens. Nor could it be explained by differences in the number of trials, because loss/stay had the fewest, and win had the most, number of trials (using the same number of trials in all conditions did not alter the pattern of results; see supplemental Information, available at www.jneurosci.org as supplemental material). Thus, there is information present in the oscillatory synchrony between the two nuclei accumbens that is not available when examining the activity of any electrode individually. In a similar finding, Tabuchi et al. (2000) demonstrated enhanced hippocampus-accumbens spike timing synchrony in rats following rewards, in absence of an increase in nucleus accumbens firing rate (although there was no punishment condition to compare with ours). Synchrony between the accumbens and Fz increased significantly compared with a prestimulus baseline level in nearly all conditions, but was not different among conditions. These results might suggest that nucleus accumbens-medial frontal cortex interactions play only a general role in feedback-related processing, although the difference in signal-to-noise between intracranial and surface EEG might have dampened any true condition differences.

Oscillations may help “glue together” functional networks not only across regions within a frequency band, but also across frequency bands within an individual region. One prominent hypothesis about the nucleus accumbens is that it gates the flow of information from limbic inputs to motor outputs (Mogenson et al., 1980; Finch, 1996; Groenewegen et al., 1996); it is possible that this mechanism is realized in part through cross-frequency coupling (O'Donnell and Grace, 1995; Brady et al., 2005). Here and in a previous study (Cohen et al., 2008b), bursts of gamma power were synchronized with the phase of simultaneously recorded α. Oscillations in the theta-α range in the nucleus accumbens may be relevant for both local coherence between medium spiny neurons and inter-neurons (Plenz and Kitai, 1998), and for long-distance coherence with the hippocampus (Tabuchi et al., 2000) and ventral tegmental area (Yavich and MacDonald, 2000). It is not clear why, in the present study, the task itself elicited strongest activity changes in the theta range, and yet gamma frequency-band power was synchronized with α. Nonetheless, gamma-α coupling is consistently observed across experiments (Cohen et al., 2008b), and thus may reflect a general mechanism of functional organization within the accumbens.

A growing number of studies is linking cross–frequency coupling to specific cognitive processes (Jensen and Lisman, 2005; Canolty et al., 2006; Cohen et al., 2008b). In the present study, gamma-α coupling diminished during loss/switch trials. This finding could not be explained by overall differences in gamma power between conditions, because there were no significant condition differences in the stimulus-locked data; thus, rather than an overall change in gamma power, loss/switch trials are characterized by a change in timing between gamma power and α phase. Indeed, it is possible that at least some of task-related gamma oscillations are synchronized not with an external stimulus, but instead with an internal brain event (e.g., slower oscillation) (Canolty et al., 2006; Cohen et al., 2008b; Tort et al., 2008). More work is needed to better understand the neural mechanisms and computational implications of gamma-α coupling, and of a break-down in cross-frequency coupling. Recently, Tort et al. (2008) found that cross-frequency coupling between the hippocampus and striatum in rats during a maze task was modulated by the rats' position along the maze. Task-related modulations of cross-frequency coupling suggest that this may be a mechanism of computation and communication within a functional network.

Cross-frequency coupling in the hippocampus is hypothesized to support the binding of multiple items within the time frame of spike-timing plasticity (Jensen and Lisman, 1996; Lisman, 2005). The physiological relevance of cross-frequency coupling in the nucleus accumbens is uncertain, but may be related to binding action-reward contingencies (which may be unbound/remapped during loss/switch trials).

We suggest that when signaling the need to adjust behavior, the functional organization of the nucleus accumbens shifts from a local to a more global, “network,” configuration, reflected in a shift from local oscillatory organization (gamma-α coupling) to larger-scale oscillatory organization (interaccumbens theta phase synchrony). This would be consistent with previous studies showing enhanced long-range oscillatory coupling during choice behavior (Pesaran et al., 2008).

Generalizability of findings is an important issue when interpreting data from patient populations. Indeed, these patients suffer from major depression, which is characterized by anhedonia. However, it is not clear that depression influenced the present results. For example, depressed individuals have decreased nucleus accumbens hemodynamic responses to positively valenced, but not to negative valenced, stimuli (Epstein et al., 2006). Other reports suggest no between-group differences in nucleus accumbens activation during simple reward motivation tasks (Knutson et al., 2008). Although it is impossible to examine nucleus accumbens electrophysiology in healthy humans, behavioral performance was similar to our previous study with a similar experimental design (Cohen et al., 2008a), and the oscillatory responses recorded at surface electrode Fz shared many characteristics with those in young healthy subjects during similar reinforcement-learning tasks (Cohen et al., 2007; Marco-Pallares et al., 2008). Furthermore, we found no significant correlations between postfeedback theta power or interaccumbens theta phase synchrony and Hamilton depression ratings at time of testing (all p > 0.1). Although this null result should be interpreted with caution because of the small number and lack of nondepressed comparison subjects, it is possible that our findings are not influenced by pathology. Indeed, although accumbens stimulation alleviates symptoms of depression (Schlaepfer et al., 2008), it is possible that DBS-driven over-stimulation of nucleus accumbens target regions (McIntyre et al., 2004; McCracken and Grace, 2007) drives the efficacy of this procedure. Consistent with this idea, stimulation of other brain regions within the same functional network, such as the subgenual cingulate, also alleviates depression symptoms (Mayberg et al., 2005). Parceling out the contribution of depression to these effects requires similar studies in rat or nonhuman primate models of depression.

Footnotes

This study was partly funded by Medtronic Inc. and a predoctoral National Institute on Drug Abuse National Research Service Award to M.XC.

References

  1. Block AE, Dhanji H, Thompson-Tardif SF, Floresco SB. Thalamic-prefrontal cortical-ventral striatal circuitry mediates dissociable components of strategy set shifting. Cereb Cortex. 2007;17:1625–1636. doi: 10.1093/cercor/bhl073. [DOI] [PubMed] [Google Scholar]
  2. Brady AM, Glick SD, O'Donnell P. Selective disruption of nucleus accumbens gating mechanisms in rats behaviorally sensitized to methamphetamine. J Neurosci. 2005;25:6687–6695. doi: 10.1523/JNEUROSCI.0643-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Canolty RT, Edwards E, Dalal SS, Soltani M, Nagarajan SS, Kirsch HE, Berger MS, Barbaro NM, Knight RT. High gamma power is phase-locked to theta oscillations in human neocortex. Science. 2006;313:1626–1628. doi: 10.1126/science.1128115. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Chrobak JJ, Buzsáki G. Gamma oscillations in the entorhinal cortex of the freely behaving rat. J Neurosci. 1998;18:388–398. doi: 10.1523/JNEUROSCI.18-01-00388.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Cohen MX, Elger CE, Ranganath C. Reward expectation modulates feedback-related negativity and EEG spectra. Neuroimage. 2007;35:968–978. doi: 10.1016/j.neuroimage.2006.11.056. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cohen MX, Elger CE, Weber B. Amygdala tractography predicts functional connectivity and learning during feedback-guided decision-making. Neuroimage. 2008a;39:1396–1407. doi: 10.1016/j.neuroimage.2007.10.004. [DOI] [PubMed] [Google Scholar]
  7. Cohen MX, Axmacher N, Lenartz D, Elger CE, Sturm V, Schlaepfer TE. Good vibrations: cross-frequency coupling in the human nucleus accumbens during reward processing. J Cogn Neurosci. 2008b;21:875–889. doi: 10.1162/jocn.2009.21062. [DOI] [PubMed] [Google Scholar]
  8. Cools R, Clark L, Owen AM, Robbins TW. Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci. 2002;22:4563–4567. doi: 10.1523/JNEUROSCI.22-11-04563.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cools R, Altamirano L, D'Esposito M. Reversal learning in Parkinson's disease depends on medication status and outcome valence. Neuropsychologia. 2006;44:1663–1673. doi: 10.1016/j.neuropsychologia.2006.03.030. [DOI] [PubMed] [Google Scholar]
  10. Deadwyler SA, Hayashizaki S, Cheer J, Hampson RE. Reward, memory and substance abuse: functional neuronal circuits in the nucleus accumbens. Neurosci Biobehav Rev. 2004;27:703–711. doi: 10.1016/j.neubiorev.2003.11.011. [DOI] [PubMed] [Google Scholar]
  11. Epstein J, Pan H, Kocsis JH, Yang Y, Butler T, Chusid J, Hochberg H, Murrough J, Strohmayer E, Stern E, Silbersweig DA. Lack of ventral striatal response to positive stimuli in depressed versus normal subjects. Am J Psychiatry. 2006;163:1784–1790. doi: 10.1176/ajp.2006.163.10.1784. [DOI] [PubMed] [Google Scholar]
  12. Finch DM. Neurophysiology of converging synaptic inputs from the rat prefrontal cortex, amygdala, midline thalamus, and hippocampal formation onto single neurons of the caudate/putamen and nucleus accumbens. Hippocampus. 1996;6:495–512. doi: 10.1002/(SICI)1098-1063(1996)6:5<495::AID-HIPO3>3.0.CO;2-I. [DOI] [PubMed] [Google Scholar]
  13. Fries P. A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn Sci. 2005;9:474–480. doi: 10.1016/j.tics.2005.08.011. [DOI] [PubMed] [Google Scholar]
  14. Genovese CR, Lazar NA, Nichols T. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage. 2002;15:870–878. doi: 10.1006/nimg.2001.1037. [DOI] [PubMed] [Google Scholar]
  15. Goto Y, Grace AA. Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nat Neurosci. 2005;8:805–812. doi: 10.1038/nn1471. [DOI] [PubMed] [Google Scholar]
  16. Groenewegen HJ, Wright CI, Beijer AV. The nucleus accumbens: gateway for limbic structures to reach the motor system? Prog Brain Res. 1996;107:485–511. doi: 10.1016/s0079-6123(08)61883-x. [DOI] [PubMed] [Google Scholar]
  17. Haber SN, Kim KS, Mailly P, Calzavara R. Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J Neurosci. 2006;26:8368–8376. doi: 10.1523/JNEUROSCI.0271-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Jensen O, Lisman JE. Hippocampal CA3 region predicts memory sequences: accounting for the phase precession of place cells. Learn Mem. 1996;3:279–287. doi: 10.1101/lm.3.2-3.279. [DOI] [PubMed] [Google Scholar]
  19. Jensen O, Lisman JE. Hippocampal sequence-encoding driven by a cortical multi-item working memory buffer. Trends Neurosci. 2005;28:67–72. doi: 10.1016/j.tins.2004.12.001. [DOI] [PubMed] [Google Scholar]
  20. Kasanetz F, Riquelme LA, O'Donnell P, Murer MG. Turning off cortical ensembles stops striatal Up states and elicits phase perturbations in cortical and striatal slow oscillations in rat in vivo. J Physiol. 2006;577:97–113. doi: 10.1113/jphysiol.2006.113050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Knutson B, Adams CM, Fong GW, Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci. 2001;21:RC159. doi: 10.1523/JNEUROSCI.21-16-j0002.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Knutson B, Bhanji JP, Cooney RE, Atlas LY, Gotlib IH. Neural responses to monetary incentives in major depression. Biol Psychiatry. 2008;63:686–692. doi: 10.1016/j.biopsych.2007.07.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lachaux JP, Rodriguez E, Martinerie J, Varela FJ. Measuring phase synchrony in brain signals. Hum Brain Mapp. 1999;8:194–208. doi: 10.1002/(SICI)1097-0193(1999)8:4&#x0003c;194::AID-HBM4&#x0003e;3.0.CO;2-C. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lakatos P, Shah AS, Knuth KH, Ulbert I, Karmos G, Schroeder CE. An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J Neurophysiol. 2005;94:1904–1911. doi: 10.1152/jn.00263.2005. [DOI] [PubMed] [Google Scholar]
  25. Lisman JE. The theta/gamma discrete phase code occuring during the hippocampal phase precession may be a more general brain coding scheme. Hippocampus. 2005;15:913–922. doi: 10.1002/hipo.20121. [DOI] [PubMed] [Google Scholar]
  26. Marco-Pallares J, Cucurell D, Cunillera T, García R, Andrés-Pueyo A, Münte TF, Rodríguez-Fornells A. Human oscillatory activity associated to reward processing in a gambling task. Neuropsychologia. 2008;46:241–248. doi: 10.1016/j.neuropsychologia.2007.07.016. [DOI] [PubMed] [Google Scholar]
  27. Mayberg HS, Lozano AM, Voon V, McNeely HE, Seminowicz D, Hamani C, Schwalb JM, Kennedy SH. Deep brain stimulation for treatment-resistant depression. Neuron. 2005;45:651–660. doi: 10.1016/j.neuron.2005.02.014. [DOI] [PubMed] [Google Scholar]
  28. McClure SM, Berns GS, Montague PR. Temporal prediction errors in a passive learning task activate human striatum. Neuron. 2003;38:339–346. doi: 10.1016/s0896-6273(03)00154-5. [DOI] [PubMed] [Google Scholar]
  29. McCracken CB, Grace AA. High-frequency deep brain stimulation of the nucleus accumbens region suppresses neuronal activity and selectively modulates afferent drive in rat orbitofrontal cortex in vivo. J Neurosci. 2007;27:12601–12610. doi: 10.1523/JNEUROSCI.3750-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. McIntyre CC, Savasta M, Kerkerian-Le Goff L, Vitek JL. Uncovering the mechanism(s) of action of deep brain stimulation: activation, inhibition, or both. Clin Neurophysiol. 2004;115:1239–1248. doi: 10.1016/j.clinph.2003.12.024. [DOI] [PubMed] [Google Scholar]
  31. Mogenson GJ, Jones DL, Yim CY. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol. 1980;14:69–97. doi: 10.1016/0301-0082(80)90018-0. [DOI] [PubMed] [Google Scholar]
  32. Mogenson GJ, Swanson LW, Wu M. Neural projections from nucleus accumbens to globus pallidus, substantia innominata, and lateral preoptic-lateral hypothalamic area: an anatomical and electrophysiological investigation in the rat. J Neurosci. 1983;3:189–202. doi: 10.1523/JNEUROSCI.03-01-00189.1983. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Mormann F, Fell J, Axmacher N, Weber B, Lehnertz K, Elger CE, Fernández G. Phase/amplitude reset and theta-gamma interaction in the human medial temporal lobe during a continuous word recognition memory task. Hippocampus. 2005;15:890–900. doi: 10.1002/hipo.20117. [DOI] [PubMed] [Google Scholar]
  34. Münte TF, Heldman M, Hinrichs H, Marco-Pallares J, Kraemer UM, Sturm V, Heinze HJ. Nucleus accumbens is involved in human action monitoring: evidence from invasive electrophysiological recordings. Front Hum Neurosci. 2008;1:2–14. doi: 10.3389/neuro.09.011.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. O'Donnell P, Grace AA. Synaptic interactions among excitatory afferents to nucleus accumbens neurons: hippocampal gating of prefrontal cortical input. J Neurosci. 1995;15:3622–3639. doi: 10.1523/JNEUROSCI.15-05-03622.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Paz R, Pelletier JG, Bauer EP, Paré D. Emotional enhancement of memory via amygdala-driven facilitation of rhinal interactions. Nat Neurosci. 2006;9:1321–1329. doi: 10.1038/nn1771. [DOI] [PubMed] [Google Scholar]
  37. Pesaran B, Nelson MJ, Andersen RA. Free choice activates a decision circuit between frontal and parietal cortex. Nature. 2008;453:406–409. doi: 10.1038/nature06849. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Plenz D, Kitai ST. Up and down states in striatal medium spiny neurons simultaneously recorded with spontaneous activity in fast-spiking interneurons studied in cortex-striatum-substantia nigra organotypic cultures. J Neurosci. 1998;18:266–283. doi: 10.1523/JNEUROSCI.18-01-00266.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Pothuizen HH, Jongen-Rêlo AL, Feldon J, Yee BK. Double dissociation of the effects of selective nucleus accumbens core and shell lesions on impulsive-choice behaviour and salience learning in rats. Eur J Neurosci. 2005;22:2605–2616. doi: 10.1111/j.1460-9568.2005.04388.x. [DOI] [PubMed] [Google Scholar]
  40. Schlaepfer TE, Cohen MX, Frick C, Kosel M, Brodesser D, Axmacher N, Joe AY, Kreft M, Lenartz D, Sturm V. Deep brain stimulation to reward circuitry alleviates anhedonia in refractory major depression. Neuropsychopharmacology. 2008;33:368–377. doi: 10.1038/sj.npp.1301408. [DOI] [PubMed] [Google Scholar]
  41. Schoenbaum G, Setlow B. Lesions of nucleus accumbens disrupt learning about aversive outcomes. J Neurosci. 2003;23:9833–9841. doi: 10.1523/JNEUROSCI.23-30-09833.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Steriade M. Grouping of brain rhythms in corticothalamic systems. Neuroscience. 2006;137:1087–1106. doi: 10.1016/j.neuroscience.2005.10.029. [DOI] [PubMed] [Google Scholar]
  43. Sturm V, Lenartz D, Koulousakis A, Treuer H, Herholz K, Klein JC, Klosterkötter J. The nucleus accumbens: a target for deep brain stimulation in obsessive-compulsive- and anxiety-disorders. J Chem Neuroanat. 2003;26:293–299. doi: 10.1016/j.jchemneu.2003.09.003. [DOI] [PubMed] [Google Scholar]
  44. Tabuchi ET, Mulder AB, Wiener SI. Position and behavioral modulation of synchronization of hippocampal and accumbens neuronal discharges in freely moving rats. Hippocampus. 2000;10:717–728. doi: 10.1002/1098-1063(2000)10:6<717::AID-HIPO1009>3.0.CO;2-3. [DOI] [PubMed] [Google Scholar]
  45. Tort AB, Rotstein HG, Dugladze T, Gloveli T, Kopell NJ. On the formation of gamma-coherent cell assemblies by oriens lacunosum-moleculare interneurons in the hippocampus. Proc Natl Acad Sci U S A. 2007;104:13490–13495. [Google Scholar]
  46. Tort AB, Kramer MA, Thorn C, Gibson DJ, Kubota Y, Graybiel AM, Kopell NJ. Dynamic cross-frequency couplings of local field potential oscillations in rat striatum and hippocampus during performance of a T-maze task. Proc Natl Acad Sci U S A. 2008;105:20517–20522. doi: 10.1073/pnas.0810524105. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Varela F, Lachaux JP, Rodriguez E, Martinerie J. The brainweb: phase synchronization and large-scale integration. Nat Rev Neurosci. 2001;2:229–239. doi: 10.1038/35067550. [DOI] [PubMed] [Google Scholar]
  48. Winer BJ. New York: McGraw-Hill; 1971. Statistical principles in experimental design: design and analysis of factorial experiments. [Google Scholar]
  49. Yacubian J, Gläscher J, Schroeder K, Sommer T, Braus DF, Büchel C. Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J Neurosci. 2006;26:9530–9537. doi: 10.1523/JNEUROSCI.2915-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Yavich L, MacDonald E. Dopamine release from pharmacologically distinct storage pools in rat striatum following stimulation at frequency of neuronal bursting. Brain Res. 2000;870:73–79. doi: 10.1016/s0006-8993(00)02403-3. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES