Local field potentials reflect dopaminergic and non-dopaminergic activities within the primate midbrain

Abstract

Midbrain dopamine neurons are thought to play a crucial role in motivating behaviors toward desired goals. While the activity of dopamine single-units is known to adhere closely to the reward prediction error signal hypothesized by learning theory, much less is known about the dynamic coordination of population-level neuronal activities in the midbrain. Local field potentials (LFPs) are thought to reflect the changes in membrane potential synchronized across a population of neurons nearby a recording electrode. These changes involve complex combinations of local spiking activity with synaptic processing that are difficult to interpret. Here we sampled LFPs from the substantia nigra pars compacta of behaving monkeys to determine if local population-level synchrony encodes specific aspects of a reward/effort instrumental task and whether dopamine single-units participate in that signal. We found that reward-correlated information is encoded in a low-frequency signal (<32-Hz; delta and beta bands) that is synchronized across a neural population that includes dopamine neurons. Conversely, high frequency power (>33-Hz; gamma band) was anticorrelated with predicted reward value and dopamine single-units were never phase-locked to those frequencies. This high-frequency signal may reflect inhibitory processes that were not otherwise observable. LFP encoding of movement-related parameters was negligible. Together, LFPs provide novel insights into the multidimensional processing of reward information subserved by dopaminergic and other components of the midbrain.

INTRODUCTION

Midbrain dopamine neurons are thought to play a crucial role in reward learning and the control of incentive motivation (Berridge and Robinson, 1998; Salamone and Correa, 2002; Wise, 2004; Schultz, 2006; Bromberg-Martin et al., 2010). The spiking activity of dopamine neurons is predicted well by the reward prediction error (RPE) signal posited by classical models of instrumental behavior (Hollerman and Schultz, 1998; Schultz, 1998; Sutton and Barto, 1998; Bayer and Glimcher, 2005) in that individual dopamine neurons respond with a burst of action potentials when a reward or reward-predictive sensory cue is more valuable than expected; but little or no change from baseline firing is evoked when reality matches the prediction (Schultz et al., 1997). After an animal learns that a particular sensory cue predicts delivery of a reward, then the size of dopaminergic responses to that cue correlates with specific attributes of the predicted reward including magnitude, probability and delay to delivery (Fiorillo et al., 2003, 2008; Tobler et al., 2005; Morris et al., 2006; Roesch et al., 2007; Joshua et al., 2008; Kobayashi and Schultz, 2008; Pasquereau and Turner, 2013, 2015). In contrast, dopamine neurons rarely encode information about the physical effort required to obtain a reward (Pasquereau and Turner, 2013). In addition to this representation at the single-unit level, it is possible for task information (reflecting e.g., RPE) to be encoded at the population level by the way that spiking is synchronized or coordinated between neurons. Synchronization of spiking in a population can influence the efficacy of signal transmission to a receiving cell population (Buzsáki, 2006). And synchrony often appears as an alignment of spikes at a characteristic oscillatory frequency (Fries, 2005; Womelsdorf et al., 2007). To date, however, few studies of dopamine neuron activity have investigated possible population-level coordination in the encoding of reward attributes or other information about the task.

Local field potentials (LFPs) recorded from the brain are thought to reflect the changes in membrane potential that are shared across a population of neuronal somata located nearby a recording electrode. These changes reflect a complex combination of spike components, after-potentials, and synaptic potentials from afferent and local circuit inputs that is difficult to disentangle (Kruse and Eckhorn, 1996; Buzsáki, 2002; Liu and Newsome, 2006; Katzner et al., 2009; Rasch et al., 2009). To interpret LFP recordings with confidence, the neural source of the signal of interest must be localized, at minimum, with respect to nucleus or brain region (Lalla et al., 2017) and, preferably, with respect to neuronal subtypes (Teleńczuk et al., 2017). A well accepted approach to LFP source localization is to construct averages of LFP signals aligned to the times of local single-unit action potentials (e.g., “spike-triggered averages”) (Goldberg, 2004). The magnitude and frequency components of a spike-triggered average provide a direct link between specific aspects of the LFP signal and a single-unit’s spiking activity.

Thus, LFP sampled from the vicinity of midbrain dopamine cells may provide insights into the population-level representation of reward and other task information in that region. Indeed, reward-associated sensory stimuli have been shown to modulate the spectral power of LFPs recorded from a variety of cortical and subcortical brain regions (Paz et al., 2006; Cohen et al., 2007, 2009; Marco-Pallares et al., 2008; Pesaran et al., 2008; van der Meer and Redish, 2009; van Wingerden et al., 2010; Lima et al., 2011; Zénon et al., 2016), suggesting that reward-dependent processes can influence the dynamic coordination of activity across populations of neurons and functional networks (Fries, 2005; Kasanetz et al., 2006; Tort et al., 2008). Notably, a study of LFPs collected from the ventral tegmental area (VTA) of the rat found that power in the theta frequency-band (4-8 Hz) increased in response to a reward-predictive cue (Kim et al., 2012). This increase was coupled with an elevated synchronization in the spiking activity of VTA dopamine neurons, supporting the view that elevated spectral power reflects, at least in part, a local synchronization of neuronal spiking within the midbrain. Because the dynamic coordination of neural activity in the midbrain between dopamine and non-dopamine neurons appears critical for encoding appetitive events (Joshua et al., 2009a; Hong et al., 2011; Okada et al., 2011; Cohen et al., 2012; Kim et al., 2012; Eshel et al., 2015, 2016), we hypothesized that potentially-distinct components of the LFP signals from that region will reflect the population activity of each neuronal type. By comparing LFPs with the spiking activities of dopaminergic single-units in the vicinity of the substantia nigra pars compacta (SNc), the present study identified dissociable oscillatory signals only one of which was related to dopaminergic single-units.

To test how LFP signals, and thus local neural synchrony, reflects reward- and effort-related information, we performed recordings in the SNc while two monkeys performed an instrumental task. Distinct instruction cues predicted the size of the upcoming reward (1, 2 and 3 drops of food) and the level of upcoming force that the animal must overcome during movement execution (0, 1.8 or 3.2 N). Our results show that reward-related events influence midbrain LFP signals in a frequency-dependent manner. Power at low frequencies (<32 Hz) correlate positively with predicted reward value, similar to the way reward is encoded by dopaminergic single-units. In contrast, LFP power at higher frequencies (>33-Hz) anticorrelates with the expected reward value. Notably, dopaminergic single-unit spiking is phase-locked only to the low-frequency components of the LFP and oscillatory encoding of movement and predicted effort is virtually non-existent. Thus, as a complement to the well-described encoding of RPE by dopaminergic single-units, our findings suggest that the synchronization of local neuronal elements also carries reward information, but in ways that differ depending on oscillatory frequency and the direct spiking involvement of dopaminergic neurons.

EXPERIMENTAL PROCEDURES

Animals

Two rhesus monkeys (monkey C: 8 kg, male; monkey H: 6 kg, female) were used in this study. Procedures were approved by the Institutional Animal Care and Use Committee of the University of Pittsburgh (protocol no. 12111162) and complied with the Public Health Service Policy on the humane care and use of laboratory animals (amended 2002). The animals were housed in individual primate cages in an air-conditioned room where water was always available. Monkeys were under food restriction to increase their motivation in task performance. Throughout the study, the animals were monitored daily by an animal research technician or veterinary technician for evidence of disease or injury, and body weight was documented weekly. If a body weight <90% of baseline was observed, the food restriction was stopped.

Apparatus

Monkeys were trained to execute reaching movements with the left arm using a two-dimensional (2-D) torqueable exoskeleton (KINARM; BKIN Technologies, Kingston, ON, Canada). Visual targets and cursor feedback of hand position were presented in the horizontal plane of the task by a virtual reality system. Distinct force levels were applied at the shoulder and elbow joints by two computer-controlled torque motors so as to simulate kinetic friction loads opposing movement of the hand. The force was constant for any hand movement above a low threshold velocity (0.5 cm/s) and directly opposed the direction of movement. After six months of training (~1000 trials per day during 125 days), both monkeys performed movements with consistent trajectories and velocity profiles under all load conditions. A plastic recording chamber (custom-machined PEEK, 28×20 mm) and head holder were fixed to the skull under general anesthesia and sterile conditions. The MRI-compatible chamber was positioned in the sagittal plane with an anterior-to-posterior angle of 20°. A detailed description of the apparatus and methods can be found in Pasquereau and Turner (2013, 2015).

Effort-benefit reaching task

Monkeys were trained to perform a reaching task that manipulated movement incentive and effort independently (Fig. 1). On each behavioral trial, the animal was required to align the cursor with two visual targets (radius: 1.8 cm) displayed in succession (Fig. 1A). A trial began when a target appeared at the start position and the monkey made the appropriate joint movements to align the cursor with this target. The monkey maintained this position for a random-duration hold period during which: (a) an instruction cue was displayed at the start position (0.7–1.3 s after hold period start, 0.5 s duration), and (b) the peripheral target was displayed (0.7–1.3 s after offset of the instruction cue, at the same location for all trials, 7.5 cm distal to the start position). At the end of the start position hold period, the start position target disappeared (the go signal), thereby cueing the animal to move the cursor to the peripheral target (within <0.8 s). Movement of the arm to capture the peripheral target required coordinated flexion at the shoulder and extension at the elbow in the horizontal plane. At a random delay after successful target capture (0.7–1.3 s), food reward was delivered via a sipper tube attached to a computer-controlled peristaltic pump (each drop = ~0.5 ml, puree of fresh fruits and protein biscuits). For the instruction cues (Fig. 1B), symbols indicated the level of upcoming force that the animal would encounter (0, 1.8 or 3.2 N) and cue colors indicated the size of the upcoming reward delivery (1, 2 and 3 drops of food). The nine unique cue types (three force levels × three reward sizes) were presented in pseudo-random order across trials with equal probability. The trials were separated by 1.5-2.5 s inter-trial intervals during which the screen was black.

Figure 1.

Effort-benefit reaching task and modulation of monkeys’ performance. (A) Timeline of the standard instrumental paradigm. A visual instruction cue was presented briefly after the animal moved the hand-controlled cursor (+) to the start position. After random delays, a target was presented (second gray circle), and then the start position was extinguished (go signal), at which time the animal was required to move the cursor to the target position. At a random delay after successful target capture, reward was delivered. (B) The instruction cue presented on a single trial was selected pseudo-randomly from 9 possible visually distinct cues. The cues were associated with different levels of reward (3 quantities of food) and effort (3 different friction loads opposing movement). (C) Measures of performance were averaged (mean ± SEM) separately for each reward/effort combination across all recording sessions (81 and 26 sessions for monkeys C and H, respectively). Reaction times and movement accelerations were affected by both expected reward value (***P<0.001, two-way ANOVA) and force level (##P<0.01, ###P<0.001, two-way ANOVA). Gray lines show the reward-related effects.

Localization of the recording site

The anatomic location of the SNc and proper positioning of the recording chamber to access it were estimated from structural MRI scans (Siemens 3T Allegra Scanner, voxel size: 0.6 mm). An interactive 3-D software system (Cicerone) was used to visualize MRI images, define the target location, and predict trajectories for microelectrode penetrations (Miocinovic et al., 2007). Electrophysiological mapping was performed with penetrations spaced 1 mm apart. The boundaries of brain structures were identified on the basis of standard criteria including relative location, neuronal spike shape, firing pattern, and responsiveness to behavioral events (e.g., movement, reward). By aligning microelectrode mapping results (electrophysiologically characterized X-Y-Z locations) with structural MRI images and high-resolution 3-D templates of individual nuclei derived from an atlas (Martin and Bowden, 1996), we were able to gauge the accuracy of individual microelectrode penetrations and determine chamber coordinates for the SNc [for anatomical details, see Pasquereau and Turner (2013)].

Recording and data acquisition

During recording sessions, a glass-coated tungsten microelectrode (impedance: 0.7-1 MOhm measured at 1000 Hz) was advanced into the target nucleus using a hydraulic manipulator (MO-95, Narishige). Neuronal signals were amplified with a gain of 10K and continuously sampled at 25 KHz (RZ2, Tucker-Davis Technologies, Alachua FL). Different bandpass filters were used to analyze the spike components (single-units: 0.3-10 kHz band pass) and the lower frequency signal (LFPs: 1-200Hz). Data was stored to disk for offline analysis. Individual spikes were isolated in Offline Sorter (Plexon Inc., Dallas TX). As for LFPs, the timing of detected spikes and of relevant task events was sampled digitally at 1 kHz. Dopamine neurons were identified according to the following standard criteria: (i) location within the SNc, (ii) polyphasic extracellular waveforms, (iii) low tonic irregular spontaneous firing rates (0.5–8 spikes/s), and (iv) long duration of action potentials (>1.5 ms).

Analysis of behavioral data

We analyzed the way the animals performed the behavioral task to test whether the behavior varied according to the levels of anticipated reward and effort. Kinematic data (digitized at 1 kHz) derived from the exoskeleton were numerically filtered and combined to obtain position, and tangential velocity and acceleration. The time of movement onset was determined as the point at which tangential position crossed an empirically-derived threshold (0.12 cm). Reaction times (RT: interval between go signal and movement onset), movement durations (MD: interval between movement onset and capture of the target), peak velocity (maximum tangential velocity), and initial acceleration (determined during the first 40 ms of the movement to avoid influences of the loads) were computed for each trial. Trials with errors (3.5% of trials) or RTs defined as outliers (4.45 × median absolute deviations, equivalent to 3 × SD for a normal distribution, 3.7% of trials) were excluded from the analysis to avoid potentially confounding factors such as inattention. Two-way ANOVAs were used to test these kinematic measures for interacting effects of incentive reward and effort.

Neuronal data analysis

Neuronal recordings were accepted for analysis based on electrode location, recording quality (signal/noise ratio of ≥ 3 SD) and duration (>200 trials). Our methods used to analyze spiking activity collected during this behavioral task have been described previously in Pasquereau and Turner (2013, 2015). For different task events (i.e., instruction cue, movement onset, and reward delivery), continuous neuronal activation functions [spike density functions (SDFs)] were generated by convolving each discriminated action potential with a Gaussian kernel (20-ms variance). Mean peri-event SDFs (averaged across trials) for each of the effort and reward conditions were constructed. A neuron’s baseline firing rate was calculated as the mean of the SDFs across the 500-ms epoch immediately preceding cue instruction. A phasic response to a task event was detected by comparing SDF values relative to a cell’s baseline firing rate (P< 0.01, t-test). A neuron was judged to be task-related if it generated a significant phasic response for at least one of the task conditions. For all single-units identified as dopamine neurons, we tested whether population-averaged activities (averaged across neurons) encoded effort and/or reward levels by using a two-way ANOVA combined with a sliding window procedure (200-ms test window stepped in 50-ms). The influence of task parameters on the activity of dopamine neurons was investigated across the full trial duration: from the epoch preceding the instruction cue to the time period after the reward delivery (total duration equals to ~7 s). To continuously visualize on figures the neuronal activity across the different task events, individual SDFs (instruction, movement onset and reward delivery) were spliced together by positioning successive events following the shortest inter-event-intervals (as the delays were randomly distributed in this task).

LFPs recorded simultaneously with spiking activities were analyzed to test for any effects of task events and effort/reward levels on oscillatory activities. Possible DC offsets, linear trends, and 60 Hz line noise were first removed. Spectral analyses of LFPs were performed using routines from the Chronux software package (www.chronux.org) for Matlab (MathWorks, Natick, MA). This open-source toolbox uses the multitaper method (Bokil et al., 2010), where data were multiplied by a specified number of orthogonal tapers (we used 5 leading tapers allowing for a frequency resolution of 1 Hz), before applying a fast Fourier transform to the tapered waveforms. The transformed spectral data were then averaged over tapers as a way of reducing variance and bias. The frequency decomposition of multitapered data segments provided a set of independent spectral estimates. This method has been successfully applied to neural data in a number of cases (Mitra and Pesaran, 1999; Pesaran et al., 2002; DeCoteau et al., 2007; McCracken and Grace, 2009; Howe et al., 2011; Wood et al., 2012; Rich and Wallis, 2017). Trial-averaged LFP power was visualized and analyzed in the form of time-frequency graphics defined across a large range of frequencies calculated between 1-120 Hz. Such spectrograms were aligned on different task events (i.e., instruction cue, movement onset, and reward) and were constructed with a series of overlapping constant-width windows of 1 second advanced in 50-ms steps. The spectrograms that show multiple alignment events were spliced together from separate spectrograms for each alignment event. The delivery of multiple drops of reward induced a temporal shift of 0.5 second per drop in animal’s valuation of reward quantity (1, 2, or 3 drops). To overcome this temporal bias, we extracted segments (a period of 0.5 second with 2 drops, and 1 second with 3 drops) in spectrograms from 1 second after the onset of reward delivery. Thus, the influence of reward quantity on LFPs followed an identical time scale, independent from the duration of the reward delivery.

For each individual frequency in the spectrogram, the pre-instruction control activity (baseline) was computed as the mean power from −1100 ms to −700 ms relative to cue onset. The LFP power was expressed in z-score units relative to baseline activity. For this, the mean power computed for the baseline activity was subtracted from the power computed in each time bin, and divided by the SD of the baseline activity. The Wilcoxon signed-rank test (P<0.01 for 4 consecutive time bins) was used to select which time-frequency bins were significantly different when comparing the baseline with the task-related activity. Then, to determine whether the LFP power was influenced by reward and/or effort levels, we performed a series of Friedman’s repeated-measures ANOVA on ranks for each time-frequency bin. The threshold for significance (P<0.001) was validated by calculating the likelihood of Type 1 (false-positive) errors during the first time bin of the pre-instruction control epoch (i.e., during a time period when the animal cannot anticipate the upcoming reward and effort levels). Non-parametric tests were used because data sets were not normally distributed (Kolmogorov-Smirnov test, P>0.05). To further characterize the effect of task parameters on LFP power, we performed a series of linear regressions to determine how the LFPs were modulated (increase or decrease) in time-frequency bins where we found a significant effect. We used the equation:

$${\text{Power}}_{\text{i}}={\beta }_{\mathit{0}}+{\beta }_{\mathit{1}}{\text{Reward}}_{\text{i}}+{\beta }_{\mathit{2}}{\text{Effort}}_i,$$

where all variables for the i^th trial were z-score normalized to obtain standardized regression coefficients (β). The sign of β values enabled to dissociate time-frequency bins where LFP power increased or decreased depending on the reward and effort levels.

To characterize the observed effects on oscillatory activities, different frequency bands were isolated in spectral power: delta (1– 4 Hz); theta (4–10 Hz); beta (14–32 Hz); gamma1 (33–59 Hz); and gamma2 (61–120 Hz). The power of each frequency band was calculated as the average value over the selected frequency bins. We performed the population-based analysis of frequency bands by comparing the grand average powers between task conditions (reward or effort levels) using the Kruskal-Wallis test.

To determine the relationship between spiking activity of dopamine neuron and the individual LFP trace simultaneously recorded, we tested whether spikes were phase-locked to LFP in a particular frequency band. The phase angle of the LFP oscillation at the time of spike occurrence was obtained by Hilbert-transforming the LFP signal band-pass filtered for the frequency-band of interest. For single-session-based analysis, the 0-360 degrees circular space (with local power maxima corresponding to 0) was divided into 12 bins of equal size, giving an angular resolution of 30 degrees. A LFP-neuron combination was considered phase-locked if the distribution of spike phase angles was non-uniform (Rayleigh’s test, P<0.001). Only segments of each recording session corresponding to periods of stronger LFP oscillations (20% of total duration) were used to compute phase-locking. For population-based analysis, the proportion of units phase-locked in various frequency-bands was plotted with a resolution of 10 degrees.

RESULTS

Before presenting results related to the neuronal recordings, we first establish that the two monkeys performed the effort-benefit reaching task correctly.

Animals’ performance

The nine conditioned stimuli presented during the task effectively communicated different levels of motivational value. This was evidenced by consistent effects on the task performance of both animals (Fig. 1C). Reaction times and kinematic measures were affected by both the expected reward magnitude (reaction times: F_(2,107)>11.27, P<0.001; movement durations: F_(2,107)>11.04, P<0.001; initial accelerations: F_(2,107)>18.22, P<0.001; maximum velocities: F_(2,107)>22.05, P<0.001) and the exerted physical effort (reaction times: F_(2,107)>5.47, P<0.01; movement durations: F_(2,107)>40.44, P<0.001; initial accelerations: F_(2,107)>34.04, P<0.001; maximum velocities: F_(2,107)>83.62, P<0.001). Although the specific effects of effort differed between reaction times and kinematics, overall movements were faster in trials in which large benefits and low efforts were anticipated. Details about effort/reward effects on monkeys’ behavior can be found in our previous study (Pasquereau and Turner, 2013).

Spiking activity of dopamine neurons

While the monkeys performed the behavioral task, we recorded wide-band signals (single-unit activity and LFPs) from 107 locations in the SNc at which putative dopamine neurons were encountered. Dopamine neurons were identified based on location, standard electrophysiological criteria and responsiveness to unexpected events. Data from the two monkeys were pooled for our analyses because no differences in response properties were found between the two animals. Although the task-evoked activities of these neurons have been described in detail elsewhere (Pasquereau and Turner, 2013; 2015), it is important to summarize those results here (Fig. 2) for comparisons with the LFP results. The neurons responded predominantly to conditioned reward-predicting stimuli as would be expected of dopamine neurons, as described classically (Hollerman and Schultz, 1998; Fiorillo et al., 2003; Tobler et al., 2005; Morris et al., 2006; Matsumoto and Hikosaka, 2009). Most dopamine neurons (78%, 83 of 107; P<0.01, t-test) emitted a phasic increase in discharge above the baseline firing rate following the presentation of instruction cues, whereas smaller fractions of cells showed a phasic response during the period preceding movement execution (46%, 49 of 107) or during the reward delivery (18%, 19 of 107). The presence of clear and sharp evoked responses (i.e., with a limited SEM) in the population-averaged activity confirmed that dopamine responses were well stereotyped around the reward-related events (Joshua et al., 2009a).

Figure 2.

Dopamine neurons encode the expected reward quantity. (A) Population-averaged activities are shown separately across task events (I: instruction, M: movement onset, R: reward delivery) for the different types of trials related to the three reward quantities or the three effort levels. The dotted lines after reward depict the three reward drops. Spike density functions (line width = SEM) were constructed with all identified dopamine neurons (n=107). (B) The effects of reward and effort on the population activity were examined using two-way ANOVA combined with a sliding window procedure (200-ms test window stepped in 50-ms). Red lines indicate different statistical thresholds (95, 99 and 99.9%). Reward encoding was detected in response to the instruction cue (F_(2,107)>7.21, P<0.001).

Consistent with the basic predictions of the RPE framework (Sutton and Barto, 1998), the magnitude of instruction-evoked single-unit responses varied with the expected reward quantity. In Figure 2A, we show population-averaged spike density functions, constructed separately across the different task events for the different types of trials related to the three reward quantities and the three effort levels. Only during the first 400-ms of the presentation of the instruction cue did the cell population firing rate encode expected reward size significantly such that the phasic cue-evoked activations scaled monotonically with the number of drops of reward (F_(2,107)>7.21, P<0.001, two-way ANOVA; Fig. 2B). In other words, the bigger the future reward, the larger the phasic response evoked following presentation of the instruction cue (Tukey–Kramer post hoc comparison). None of the population responses triggered by other task events (i.e., movement and reward delivery) encoded information about reward size (F_(2,107)<2.66, P>0.05, two-way ANOVA; Fig. 2B). Furthermore, no population responses were influenced by the level of effort (Fig. 2B bottom), neither around the time it was predicted by the instruction cues, nor when it was exerted by the animal during movement execution (F_(2,107)<0.44, P>0.05, two-way ANOVA). Our previous cell-by-cell analysis identified a small subset of dopamine neurons (11%, n=12/107; Pasquereau and Turner, 2013) whose cue-evoked responses encoded predicted physical effort robustly and in a manner that closely matched behavioral performance. The absence of significant effort-related modulations in the population-level analysis performed here (Fig. 2B) confirms that encoding of effort-related information was rare in our population of dopamine single-units.

Phase locking between LFP and spiking activity

LFP data from the same 107 recording sessions were used to investigate any link between the encoding of task information by neural spike trains and LFP oscillatory activities. Raw LFP signals were split into frequency-bands of interest by band-pass filtering the signal. Delta, theta, beta, gamma1, and gamma2 rhythms were observed to vary across the successive events of the task. To determine whether these LFP signals could be attributed to local neuronal processes, we tested whether spikes were phase-locked to particular frequency bands of the LFP. As shown for two examples in Figures 3A-B, significant phase-locking was revealed by the non-uniform distribution of the LFP phase angles at the time of spike occurrence (Rayleigh’s test, P<0.001). We found that almost all neurons (99%, 106 of 107; Fig. 3C left) were phase locked to the delta rhythm, with preferred phases between 330 and 0 degrees, indicating that the probability of dopamine spiking was highest just before the local peak in delta oscillation. A smaller proportion of neurons (33%, 35 of 107; Fig. 3C right) showed spike-LFP phase locking in the beta frequency range. In this case, consistent with a previous primate study (Bartolo et al., 2014), the preferred phase ranged between 150 and 180 degrees, indicating that action potentials tended to occur just before the local trough in oscillation. A third minor category (7%, 8 of 107) of significant phase locking was found in the theta frequency range, although that level of incidence was not significantly greater than chance (χ²=0.54, P=0.46; χ² test). No neurons showed spike-LFP phase relations in the gamma frequency bands. Thus, while we did not record simultaneously from pairs of neurons, the high incidence of significant phase locking to delta and beta frequencies and the narrow distribution of preferred phases for that locking (Fig. 3C) implies that the spiking activity of many individual dopamine neurons can potentially contribute to low frequency (<32-Hz) components of LFP signals recorded from the SNc. Next we tested whether these LFP low frequency signals encode reward-related information similar to the RPE encoding observed in the spiking activity of dopamine single-units.

Figure 3.

Phase locking of spike discharge of dopamine neurons. (A-B) Two exemplar neurons that were classified as phase locked to the (A) delta rhythm and (B) the beta rhythm. Each spike-triggered average and circular plot depicts the relationships between the spike occurrences and the filtered LFP signals. Significant phase-locking was revealed by the non-uniform distribution of the LFP phase angles at the time of spike occurrence (Rayleigh’s test, P<0.001). The arrow in individual circular plot shows the mean preferred angle. (C) Population results in which the number of units (in %) that were significantly phase-locked is shown as a function of the preferred angle detected (in degrees). The number of units phase-locked to LFP signals and the distributions of the preferred phases obtained with delta/beta rhythms indicate that dopamine neurons were well coordinated by these relative low-frequency oscillations.

Task-dependent LFP oscillations

We computed LFP spectrograms aligned on different task events (i.e., instruction, movement onset and reward; 1-120 Hz) for each individual recording session. In Figure 4A, we show the normalized LFP power averaged across all recording sessions (n=107) and across different task conditions (three reward quantities and three effort levels). Task-dependent changes in population-averaged LFP power were evident throughout the range of frequencies examined. For the population-averaged activity, we observed significant deviations of oscillatory power away from baseline during successive behavioral periods (threshold: P<0.01 for 4 consecutive time bins, Wilcoxon signed-rank test). The initial change in LFP power was a small decrease in anticipation of the instruction cue followed by a large increase in power immediately following presentation of the instruction cue. A second decrease in LFP power appeared immediately prior to reward delivery followed by a second large increase beginning approximately two seconds later. Notably, anticipatory decreases in LFP power preceded both reward-related events (instruction cue and reward delivery) and these decreases were deepest in the beta frequency-band (14-32 Hz).

Figure 4.

The effort-benefit task modifies the power of LFP signals. (A) Population-averaged LFP spectrogram aligned on successive task events (i.e., I - instruction, M - movement onset and R - reward). LFP powers were averaged across all recording sessions (n=107) and across different task conditions (three reward quantities and three effort levels). Power is color-coded on a normalized scale calculated from the control period (−1100 to −700 ms relative to instruction). Distinct task-dependent changes in LFP signals were detected: white and black lines delineate significant decrease and increase in power, respectively (Wilcoxon signed-rank test; P<0.01 for 4 consecutive time bins). (B) Spectral powers (mean ± SEM) of different frequency bands were compared between animals (81 versus 26 sessions for monkey C and H, respectively). (C) The inter-individual variability in the spectral powers was exanimated using the Kruskal-Wallis test. As in Fig. 2, red lines indicate different statistical thresholds.

To test whether similar task-dependent changes in LFP were observed in both monkeys, we compared spectral powers of different frequency bands between data sets from the two animals (Kruskal-Wallis test; Figs. 4B-C). Using a statistical threshold of 99.9%, no difference in LFP power was observed between animals during instruction, movement or post-reward periods. A difference between animals was found however in the beta frequency range (χ²>10.96, P<0.001; Fig. 4C) during a short period preceding the reward delivery. In subsequent analyses that combined data across animals, caution was exercised when a task-dependent effect was investigated in beta rhythm around the pre-reward period.

Influence of reward level on LFP

As described before, the spiking activity of dopamine neurons encoded the quantity of reward predicted by the instruction cue. We tested whether LFP oscillations were similarly influenced by the reward level. In Figure 5A, we show population-averaged spectrograms constructed separately for trials with small, medium, and large reward quantities. A series of Friedman’s repeated-measures ANOVAs on ranks was performed to identify individual time-frequency bins where the LFP power was correlated with reward quantity (χ²>13.82, P<0.001; Fig. 5B). Overall, we found that reward size had some influence on the LFP signal in 60% of recording session (64 of 107). Across the course of behavioral trials, these influences were most common during and immediately following appearance of the instruction cue (significant reward-dependent modulation of LFP in 30% of sessions, Fig. 5B red line) and following reward delivery (32% of sessions). For the time-frequency bins judged to be reward-dependent, we then estimated the relation between LFP power and reward size by measuring the slope of a regression line fitted to the data collected during different trial types. As shown in Figure 5C, reward-dependent changes in LFP power differed markedly depending on LFP frequency and the task period. In specific low frequency bands (<32 Hz), LFP power increased in concert with reward level, both during the instruction period, transiently, and following reward delivery. In contrast, LFP power across a wide range of high frequencies (>33 Hz) varied inversely with reward level and this relationship was present throughout most of the interval between instruction cue appearance and reward delivery and then again following reward delivery. The relative time courses of these reward-dependent variations in LFP power are illustrated in Figure 6A. Population-averaged LFP power within frequency-bands of interest was estimated separately for the three reward levels and compared using a series of Kruskal-Wallis tests (χ²>14.03, P<0.001; Fig. 6B). Following presentation of the instruction cue, the population LFP response was positively correlated with the predicted reward size first in the theta frequency range and then, ~200-ms later, in the beta band. Following this, a negative reward-dependent relationship (i.e., with a negative regression coefficient) emerged in the high frequencies of gamma2 and then attenuated before movement initiation. No significant reward-dependent modulations in LFP power were detected during motor phase of the task between movement onset and reward delivery. Approximately two seconds after reward delivery, positive reward-dependent relationships appeared in all frequency bands above theta, with the strongest relationship in the beta frequency range.

Figure 5.

Oscillatory power was modulated by reward level. (A) Population-averaged LFP spectrograms constructed separately for the different types of trials related to the three reward quantities (low, medium and high). These figures follow the conventions of Fig. 4A. The black triangle on the top indicates when a temporal segment was extracted from the data to get similar time scale between conditions after the reward delivery. (B) Influence of reward levels on LFP power was detected by a series of Friedman’s repeated-measures ANOVA (χ²>13.82, P<0.001). For each time-frequency bin, the gray scale represents the fraction of sessions significantly modulated. The red line indicates the maximum fraction of sessions modulated by the reward level across all oscillatory frequencies. (C) LFP power either increased (yellow: positive regression coefficients) or decreased (blue: negative coefficients) as a function of the reward level (linear regression).

Figure 6.

The time course of reward-dependent changes in LFPs. (A) The population-averaged signal (mean power ± SEM) was decomposed into oscillatory frequency-bands of interest for the different types of trials related to the three reward quantities. (B) Reward-dependent changes in the population averages were tested by a series of Kruskal-Wallis test. As in Fig. 2, red lines indicate different statistical thresholds. Tukey-Kramer post hoc comparisons were used to determine the type of tuning [positive (yellow) or negative (blue) slope].

Together, we found that LFP power encoded the size of an anticipated or delivered reward in a dynamic way that differed across frequency bands and task epochs. Power in the theta and beta ranges correlated positively with reward size, roughly similar to the positive relationship described for dopamine single-units (Fig. 2). Moreover, these changes appeared at short latency relative to the behavioral event (especially following instruction delivery) and lasted transiently, which is somewhat similar to the phasic responses observed in dopamine single-units. Conversely, the inverse relationship to reward size observed at higher frequency bands (> 32-Hz) appeared at a longer latency following instruction appearance and reward delivery and persisted longer, all of which contrasts sharply with the phasic positive-encoding responses of dopamine neurons. It is important to note, however, that the population activity of dopamine single-units did not encode reward size following reward delivery (Fig. 2B) as reinforcement theory predicts for such a condition in which there is no reward prediction error (Sutton and Barto, 1998). In contrast, LFP power during this period encoded reward size robustly across a wide range of frequencies above the theta frequency band. Thus, while dopamine neurons signaled a RPE (Schultz et al., 1997; Bayer and Glimcher, 2005), the LFP power varied as a function of the actual reward received. Although we cannot exclude the possibility that this relationship was secondary to some aspect of behavior that, itself, covaried with reward size, we observed no overt behaviors in our animals fit with that explanation.

Influence of effort level on LFP

In parallel, to investigate whether LFP signals were influenced by the anticipation and the exertion of physical effort during movement execution, we used the same methods as described above. In Figure 7A, the population-averaged spectrograms were constructed separately for trials that used the three levels of effort. We found that the effort level influenced only 5% of LFP recordings (5 of 107 sessions), and this mainly during movement execution when the different forces were applied (Figs. 7B-C). Because the incidence of this effort-dependent effect on LFP population was not significantly greater than chance (χ²=0.01, P=0.9; χ² test), we considered such influence as negligible. Consistent with this result, the population-averaged analysis of frequency-bands showed that no specific rhythm of LFP oscillations was significantly modulated by the different friction loads (χ²<12.87, P>0.001, Kruskal-Wallis test; Fig. 8). Although we observed a trend to encode the effort level during the movement execution in the beta frequency range, it remained insignificant when we used the same statistical threshold as used in the analysis of reward level encoding (99.9%).

Figure 7.

Oscillatory activities were insensitive to the effort level. (A) Population-averaged LFP spectrograms constructed separately for the different types of trials related to the three effort levels (low, medium and high). These figures follow the conventions of Fig. 5. (B-C) Influence of effort levels on LFP power (Friedman’s repeated-measures ANOVA, P<0.001). Note that the effort level influenced a non-significant proportion of LFP recordings (5 of 107 sessions; χ2=0.01, P=0.9).

Figure 8.

The time course of effort-dependent changes in LFPs. (A) The population-averaged signal (mean power ± SEM) was decomposed into oscillatory frequency-bands of interest for the different types of trials related to the three effort levels. (B) Effort-dependent changes in the population averages were tested by a series of Kruskal-Wallis test. As in Fig. 2, red lines indicate different statistical thresholds.

DISCUSSION

Single-unit recordings performed in behaving non-human primate have largely shown how the midbrain dopamine neurons play a crucial role in reward learning, attention, temporal prediction, sensory processing and motivating behaviors toward desired goals (Romo and Schultz, 1990; Schultz and Romo, 1990; Ljungberg et al., 1992; Schultz, 2006; Bromberg-Martin et al., 2010; Nomoto et al., 2010; de Lafuente and Romo, 2011; Soares et al., 2016). To date, however, LFPs collected within the vicinity of these cells has remained surprisingly under-explored.

The present results reveal that LFP oscillations can be used to investigate reward-related processes within the midbrain and observe the dynamic coordination of components related and unrelated to the spiking of dopamine neurons. Despite some differences with dopamine spiking activity, in general we found that the power of LFP oscillations was tuned transiently by reward-related events during the task performance, predicting and responding to the reward delivery while disregarding processes associated with movement execution. Because a fraction of dopamine neurons were phase-locked to the delta and beta rhythms, low-frequency bands appeared as the only marker of the activity of local dopamine cells among the LFP signals. In support of this view, low-frequency bands (< 32-Hz) responded to the reward-predictive cue by scaling their power as the dopamine neurons modulate their firing rate. Indeed, consistent with the basic predictions of reinforcement theory (Sutton and Barto, 1998), the low-frequency signals encoded as well the expected reward value by gradually increasing power as a function of the quantity of the upcoming reward. The time course of those reward-dependent modulations in power correlated relatively well with the encoding of reward by dopamine single-units. For the high-frequency oscillations (> 33-Hz), instead, we found no evidence for direct relationships to dopamine spiking activity. Converse to the encoding by dopamine neurons and low-frequency LFPs, the greater power of the instruction-evoked gamma oscillations the smaller the expected reward quantity. Moreover, this reward-dependent effect began at a long latency, removing it in time from the coordinated population response of dopamine neurons. We speculate that this high-frequency LFP power may reflect synchronized activity of inhibitory inputs to the SNc (Buzsáki and Wang, 2012), while it is also quite possible that these oscillatory signals result of volume-conduction from nearby structures such as the substantia nigra pars reticulata.

One of the most notable differences that we found between LFP oscillations and dopamine neuron activity was in the encoding after reward delivery. While LFP power was modulated as a function of the actual size of reward received, the dopamine neurons as a population did not respond, consistent with their encoding of an RPE-like signal (Schultz et al., 1997; Bayer and Glimcher, 2005) (Fig. 2). This difference in encoding could reflect a local transformation of information from the afferent information received by dopamine neurons (reward size reflected in low-frequency LFPs) to that transmitted (RPE reflected in the spiking of dopamine neurons). While we did not observe any consistent behavioral factor to explain such results, we cannot exclude the possibility that LFP power varied indirectly instead as a function of different behavioral patterns (e.g., eye movements, chewing the food reward) triggered by different size of reward received.

There is considerable evidence that midbrain dopamine neurons respond consistently with a phasic burst of activity to different types of appetitive events including primary rewards and reward-predictive stimuli (Romo and Schultz, 1990; Schultz and Romo, 1990; Ljungberg et al., 1992; Wise, 2004). Those dopamine responses closely match the RPEs posited by classical models of reinforcement learning (Hollerman and Schultz, 1998; Schultz, 1998; Sutton and Barto, 1998; Bayer and Glimcher, 2005). As a facet of that prediction error signal, dopamine responses encode the reward value predicted by conditioned stimuli by combining multiple attributes of the expected reward such as the quantity, the probability and the time delay to delivery (Fiorillo et al., 2003, 2008; Tobler et al., 2005; Morris et al., 2006; Roesch et al., 2007; Joshua et al., 2008; Kobayashi and Schultz, 2008; Pasquereau and Turner, 2013, 2015). In the present study, we confirmed that midbrain dopamine neurons play a role in motivating behaviors by encoding the expected size of the upcoming reward such that the phasic cue-evoked activations scaled monotonically with the quantity of reward (Fig. 2). The strength and the sharp profile of cue-evoked activities observed in our population-averaged analysis indicated that dopamine responses were stereotyped around the reward-related event. This tight temporal linkage between spiking activities has been widely observed for dopamine neurons, which, classically, have been described as a homogenous system (Eshel et al., 2016) despite some distinctions based on gene expression, electrophysiological properties, and afferent inputs (Haber et al., 1995; Neuhoff et al., 2002; Ford et al., 2006; Margolis et al., 2008; Brown et al., 2009; Lammel et al., 2012). Considering this, we wondered if the coordination of activity across a population of dopamine neurons, as reflected in LFP recorded in their vicinity, also carries task information. Here, we found evidence consistent with that hypothesis in that dopamine encoding was accompanied by low-frequency LFP signals (i.e., theta and beta) whose power also increased as a function of the expected reward value (Figs. 5–6). Unfortunately, our approach did not permit a causal test of the rolls low-frequency rhythmic activity play within the neuronal assembly. Interestingly, increased theta power coupled with a phase synchrony of dopamine neurons has already been reported in the ventral tegmental area when a reward-predictive stimulus was presented to a rat (Kim et al., 2012). During the acquisition of cue-reward association, the proportion of these coordinated activities phase-locked to the theta rhythm correlated with the conditioned value associated to the cue (Joshua et al., 2009a; Kim et al., 2012), suggesting that low-frequency signals may be an important component of the dynamic processes related to reward conditioning. In addition to a specific role in the local processing of reward prediction, theta rhythms have been proposed to support long-range synchronization across neuronal networks to facilitate communication between brain regions (Buzsáki and Draguhn, 2004; Siapas et al., 2005; Park and Moghaddam, 2017). Thus, the tuning of low-frequency oscillations observed in our study (Figs. 5–6) may potentially serve to synchronize the local dopamine spiking activities during the period of reward anticipation and may be also reflective of inter-region processing necessary for coding value signals (van Wingerden et al., 2010). Further studies are needed to assess the exact contribution of LFP signals in the coordinated activity of both local and distal neuronal networks.

In complement, we found that high-frequency oscillations in the midbrain were modulated by reward-predictive cue such that the gamma power was inversely correlated with the associated reward value (Figs. 5–6). In other words, high-frequency oscillations preferentially responded to a low appetitive conditioned stimulus rather than to a cue associated with high reward quantity. This result contrasts with the instruction-evoked responses of dopamine neurons, which increased as a function of reward size. Although cortical studies have reported that high-frequency signals are tightly coupled to the firing of local neurons (Liu and Newsome, 2006; Belitski et al., 2008; Ray et al., 2008), here we observed no direct link with the dopamine spiking activity, suggesting that this gamma signal reflects non-dopamine components within the nearby neuronal elements. As the current dataset only contained dopamine-like single-units, no direct evidence exists as to the origin of the high-frequency oscillations. However, because gamma power was anti-correlated with reward size, it is tempting to hypothesize that the signal originated from local or inter-regional inhibitory elements. First, in addition to dopamine neurons, ~30% of neurons in the SNc are γ-aminobutyric acid (GABA)-ergic neurons (Nair-Roberts et al., 2008; Creed et al., 2014). These local GABAergic neurons strongly inhibit dopamine neurons (Tan et al., 2012), are implicated in reward prediction with increased firing in response to conditioned stimulus (Cohen et al., 2012), and contribute directly to computation of RPEs via a subtraction (Eshel et al., 2015). We speculate that these GABA neurons increase their discharge rate mainly for low-reward values as described here for high-frequency oscillations. To our knowledge, however, no study has described such a GABA response pattern in the midbrain. Second, the high-firing rate GABAergic neurons of the nearby substantia nigra pars reticulata might also contribute to this gamma LFP signal, but the predominant responses in these neurons occur when the cues predict a high reward value (Joshua et al., 2009b). Third, midbrain dopamine neurons receive massive GABA inputs from several more distant regions such as the striatum, the pallidum and the rostromedial tegmental nucleus (RMT) (Haber et al., 1990; Jhou et al., 2009). Of particular interest is that RMT activity encodes rewards in a pattern similar to that of the high-frequency LFP observed here and inverse to that of dopamine neurons (Hong et al., 2011). Further work is required to clarify which components within the neuronal network are linked directly to LFP high-frequency oscillations.

Midbrain dopamine neurons signal the RPEs posited by classical models of reinforcement learning (Hollerman and Schultz, 1998; Schultz, 1998; Sutton and Barto, 1998; Bayer and Glimcher, 2005). In our paradigm, the instruction cues informed the animal about the exact future quantity of reward to be delivered at the end of the trial. Thus, consistent with previous observations (Schultz et al., 1997), little or no dopamine single-unit response was evoked by the food reward because the reality matched to the prediction (Fig. 2). Contrary to single-unit activities, however, we found that the majority of LFP frequency-bands recorded in the SNc, and the beta band power in particular, were influenced by the trial outcome (Figs. 5–6). After reward delivery, LFP power correlated with the actual reward received rather than reflecting an RPE-like signal. This result reinforces the view that LFP power does not simply reflect the population-level activity of local dopamine single-units. If this phenomenon does not result from some unseen behavioral covariate, a credible neuronal substrate for such LFP signal could be the afferent projections from the pedunculopontine nucleus (PPN) to the SNc. PPN neurons make glutamatergic and cholinergic synaptic connections in the midbrain (Scarnati et al., 1986; Futami et al., 1995; Mena-Segovia et al., 2004) that provide one of the strongest sources of excitatory input to dopamine neurons (Matsumura, 2005; Pan and Hyland, 2005). In the primate, one category of PPN neurons has been shown to discharge phasically after reward delivery with a level of activity associated with the actual reward (Okada et al., 2011). PPN neurons exhibited stronger firing for large reward than for small one (Okada et al., 2009), thus mimicking our oscillatory signals. Taken together, LFP oscillations collected in the SNc may provide additional data dimensions along which distinct processing of predicted and actual reward values differ, both of which are necessary for the computation of RPE as represented by dopamine neurons.

In Pasquereau and Turner (2015), we have reported that even though a large fraction of our dopamine neurons changed their firing prior the time of movement onset (see Fig. 2), the timing of those responses was linked to the appearance of the visual go-signal and their magnitude did not correlate with any of a variety of measures of motor performance. Although consistent with the classical view (DeLong et al., 1983; Schultz et al., 1983), our observations were at variance with recent studies that described a direct link between the activity of midbrain dopamine neurons and motor performance (Jin and Costa, 2010; Barter et al., 2014, 2015; Varazzani et al., 2015; Dodson et al., 2016; da Silva et al., 2018). Those discrepancies raised the possibility that a select subset of dopamine neurons, one that we failed to sample from, may encode motor information. Because LFP reflects the summation of electrical fields originating from within a small (~250μm) distance of the electrode contact (Katzner et al., 2009), the present analyses offered the opportunity to overcome possible biases in the single-unit data collection (Towe and Harding, 1970). However, consistent with our initial findings, we found no evidence for a modulation of LFP power related to movement execution (Fig. 4A). None of the frequency-bands were modulated significantly by the effort level while the monkeys ‘performance varied depending on friction loads (Figs. 7–8). These results expand our previous cell-by-cell analyses (Pasquereau and Turner, 2013, 2015) in failing to find a signal in the midbrain that encoded movement or kinematics during our instrumental task.

As it is notoriously difficult to record from dopamine neurons due to their sparsity and depth, we demonstrated that LFP recordings and oscillatory activities described herein could provide an alternative tool to investigate the synchronized activities within the midbrain. We found that reward-related changes in low frequency-bands could be used to reveal a response pattern similar to dopamine spiking activity. But also, we observed that non-dopamine signals were reflected in the power of different oscillatory frequency-bands, providing a method for sampling the activity from diverse population of neurons. To determine how the midbrain processes information related to appetitive behaviors it will be essential to understand the dynamics of interactions between local and afferent signals. Novel methods using optogenetic or chemogenetic tools (i.e., the selective inhibition or stimulation of cell types) in concert with LFP recordings should be used to identify cell populations and their functions unambiguously. LFP recordings appear also relevant to future research concerning the pathophysiology of neurological and psychiatric disorders. For example, impairment of coordinated activity and joint processing of information in the midbrain may be one mechanism that produces the symptoms of disorders such as schizophrenia, for which there is no consistent dopamine marker (Moghaddam and Wood, 2014).