Reward magnitude tracking by neural populations in ventral striatum

Abstract

Evaluation of the magnitudes of intrinsically rewarding stimuli is essential for assigning value and guiding behavior. By combining parametric manipulation of a primary reward, medial forebrain bundle (MFB) microstimulation, with functional magnetic imaging (fMRI) in rodents, we delineated a broad network of structures activated by behaviorally characterized levels of rewarding stimulation. Correlation of psychometric behavioral measurements with fMRI response magnitudes revealed regions whose activity corresponded closely to the subjective magnitude of rewards. The largest and most reliable focus of reward magnitude tracking was observed in the shell region of the nucleus accumbens (NAc). Although the nonlinear nature of neurovascular coupling complicates interpretation of fMRI findings in precise neurophysiological terms, reward magnitude tracking was not observed in vascular compartments and could not be explained by saturation of region-specific hemodynamic responses. In addition, local pharmacological inactivation of NAc changed the profile of animals’ responses to rewards of different magnitudes without altering mean reward response rates, further supporting a hypothesis that neural population activity in this region contributes to assessment of reward magnitudes.

INTRODUCTION

Animals enact behavioral strategies largely based on the amount of positive reinforcement or reward they expect to receive from competing courses of action (Doya, 2008; Kringelbach and Berridge, 2009). Rewards come to be associated with actions, as well as with environmental stimuli, and give rise to their perceived values. Associations are learned and values adjusted by comparing earned and expected rewards in each behavioral context (Sutton and Barto, 1981). When choosing from a set of options, animals will often allocate their decisions in direct proportion to the reward magnitude associated with each option; two equally rewarding options will be chosen with equal probability, and more rewarding options will be chosen more frequently (Herrnstein, 1970). A requirement in most types of reward-related behavior is therefore that animals have a way to assess rewards in at least semi-quantitative fashion. Most fundamentally, animals must be able to evaluate the magnitudes of intrinsically rewarding stimuli (primary rewards) in much the same way they evaluate sensory variables such as luminance, texture, and tone.

Neurons with firing rates dependent on reward magnitudes have been identified in the prefrontal cortex, striatum, amygdala, and dopaminergic midbrain (Schultz, 2015). Some of these neurons fire in absolute proportion to reward magnitude, immediately after delivery, but most have more complex dependence on behavioral variables, including the range of reward magnitudes presented in a task and the presence of stimuli predictive of rewards (Schultz, 2000). Dopaminergic fibers that project to the nucleus accumbens (NAc) have often been considered essential to reward processing, and are among those that signal when rewards are anticipated as well as experienced (Saddoris et al., 2015). The functional relationships among reward-responsive neurons in different brain regions and their relationship to broader neuronal networks remain a topic of intense interest. Because studies of reward are typically performed using electrophysiology or electrochemistry in awake, behaving animals, it is hard to isolate low-level neuronal events involved in integrating rewarding stimulus inputs from higher-level processes that presumably relate reward magnitudes to other stimuli or task components; it is also difficult to compare different brain regions or cell populations to assemble a comprehensive picture of neural tuning to rewards, analogous to maps of functional architecture obtained in sensory systems.

Neural stimulation methods permit the dissociation of reward delivery from other aspects of behavior, and could be particularly useful in defining mechanisms of reward magnitude computation. A robust, clinically-relevant, and empirically well-characterized technique involves electrical microstimulation of various brain regions, most prominently including the medial forebrain bundle (MFB) (Olds and Milner, 1954). The trade-off between the stimulation strength and duration of pulse trains required to support any given level of operant responding for MFB stimulation has led to the suggestion that action potentials triggered by the stimulation are summed by a neural integrator closely associated with the animal’s assessment of reward magnitude and consequent behavioral performance (Gallistel, 1978; Gallistel et al., 1981; Sonnenschein et al., 2003). Importantly, MFB stimulation-mediated reward has been shown to substitute for sucrose solutions in behavioral tasks (Conover and Shizgal, 1994), implying that the circuitry activated by MFB stimulation coincides in functionally relevant ways with the neural pathways required for processing naturalistic rewards. Unlike many naturalistic rewards, however, MFB stimulation reward is easily combined with high resolution noninvasive brain imaging, and has been used by ourselves and others in recent MRI studies (Krautwald et al., 2013; Lee et al., 2014).

In an effort to discern essential components of reward magnitude processing at a whole-brain level, we combined MFB stimulation reward with behavioral psychophysics, local pharmacological inactivation, and functional magnetic resonance imaging (fMRI) in rats. This combined approach allowed us to manipulate the reward system and record quantitative neurophysiological information from distributed neural populations. We were able to survey the entire brain for foci of reward magnitude tracking—areas whose fMRI signals track psychometric reward magnitude measurements—which could then be probed with targeted brain inactivation to test whether spatially-distinct neural populations play a role in reward magnitude integration.

METHODS

Implantation of stimulation electrodes and cannulae

All surgical and animal handling procedures were performed in accordance with federal and institutional guidelines, and were approved by the MIT Committee on Animal Care. Adult male Lewis rats (250–300 g) were implanted with stimulation electrodes in the medial forebrain bundle (MFB) at the level of the lateral hypothalamus (LH). Monopolar stimulating electrodes were fabricated from 0.063 mm-diameter teflon-coated silver wire (A-M Systems, Sequim, WA), cut to lengths of approximately one centimeter. These electrodes were comparable to monopolar electrodes used in previous MFB stimulation studies most relevant to our experiments, and the use of silver wire, compared with alternative materials, minimized the MRI artifact associated with magnetic susceptibility of the electrode. Approximately 0.5 mm of the insulation was stripped from one end of each electrode to form a tip. The length of the wire (63.5 mm) was threaded through 0.762/1.587 mm inner/outer (ID/OD) diameter polyetheretherketone (PEEK) tubing (McMaster-Carr, Robbinsville, NJ) and connected to one pin of a two-pin connector (Digi-Key, Thief River Falls, MN). The second pin was connected to a bare silver reference wire of 63.5 mm length and 0.125 mm diameter.

Animals were anesthetized and underwent surgery to introduce craniotomies for electrode insertion, 2.2 mm, 2.0 mm or 1.2 mm posterior to bregma and 1.7 mm left of the midline suture. Electrodes were lowered through the craniotomy to a depth of 8.6 mm below the skull surface. An additional hole was drilled through the skull for introduction of a conducting, beryllium copper screw (Antrin Enterprises, Ojai, CA); the stimulating electrode ground wire was wound around this screw and attached with silver paint. Some animals were further implanted with a 2.0 or 7.5 mm long 22 GA PEEK guide cannula (Plastics One, Roanoke, VA) for the infusion of lidocaine into NAc. Guide cannulae were fitted with a Nylon dummy cap with a stylet that protruded 0.5 mm beyond the end of the guide to prevent blockage of the cannulae. A custom-made polyetherimide head post, tapped to accept two nylon screws from the MRI coil mounting head gear, was cemented to the dorsal surface of the skull anterior to the stimulating electrode for animals undergoing fMRI. Dental cement was applied to the entire skull surface area to hold the implants rigidly in place. Electrode and cannula positions were confirmed using MRI data and histologically confirmed in a subset of rats. For several animals, electrode impedances were measured before each behavioral session using an FHC Impedance Conditioning Module (FHC Inc., Bowdoin, ME); all measured animals displayed stable electrode impedances in the range from 4.9–6.6 kΩ.

Behavioral techniques

Electrode-implanted animals underwent behavioral shaping and measurement procedures. All behavioral experiments were performed in a plexiglass operant chamber (28 × 21 × 21 cm; Lafayette Instruments, Lafayette, IN), placed in a lighted sound-proof cabinet (Med Associates, St. Albans, VT). Two infrared nose poke sensors (Med Associates) were positioned at one end of the operant chamber, 5 cm from the floor, and a light emitting diode (LED) indicator was positioned near the top of the chamber above the sensors. Input from the nose poke sensors was monitored by a laptop computer via a digital input/output interface (National Instruments, Austin, TX). A custom-written computer program was used to time output stimuli dependent on the detected nose pokes. Output pulses from the computer were used to trigger cathodal stimulus pulses (0.2 ms) delivered at frequencies from 44–386 Hz by a constant current isolated stimulator (World Precision Instruments Isostim A320R, Sarasota, FL). The applied frequencies are in the standard range used to evoke MFB stimulation reward-related behavioral effects in animals, and are also comparable to clinically relevant deep brain electrical stimulation frequencies, which are typically delivered at over 100 Hz. Poles of the stimulator were shorted to one another in between stimulus pulses to prevent charge buildup on the electrodes, and pulses were visualized on an oscilloscope to verify consistent amplitude and pulse shape.

Animals were initially shaped to perform nose pokes to elicit MFB stimulation. In a typical experiment, an animal was rewarded for each nose poke with a one second 150 Hz train of 0.2 ms pulses, delivered at the maximum stimulus current (0.3–0.9 mA) for which no overt motor artifact was observed. The minimum interstimulus interval was 0.5 s. Shaping sessions lasted 30 minutes to one hour, once per day, for 2–10 days. Animals that displayed fewer than ~20 nose pokes per minute were eliminated from the study. Following shaping, psychometric “reward titration” curves were measured using a two choice operant task. The experimental procedure was based on studies of Gallistel and others (Gallistel and Leon, 1991; Mark and Gallistel, 1993; Simmons and Gallistel, 1994), who have shown that rats prefer rewarding stimuli of increasing intensity, up to a saturation point beyond which more intense stimulation is indistinguishable from the saturating reward.

Rats destined for imaging experiments were placed in an operant chamber with two nose poke sensors, and their behavior was monitored over a series of five-minute trials. Triggering of either sensor during a trial elicited a 1 s pulse train (0.2 ms pulses) of MFB stimulation, delivered with currents determined as described above and minimum inter-train interval of 0.5 s. Poking into one sensor elicited stimulation at a fixed “reference” frequency of 150 Hz, found to be above the saturation frequency for all eight individuals included in our initial reward titration analysis. Poking into the other sensor elicited stimulation at a second, varied “comparison” frequency. The comparison stimulus was delivered at 44, 53, 64, 77, 93, 111, 134, 161, 193, or 231 Hz. For all animals, at the start of each behavioral session, a dummy trial was conducted to reshape animals to the task. Then ten trials were performed, in each of which an animal compared one of the ten comparison stimuli to the fixed reference stimulus. Trials using the ten different comparison frequencies were performed in random order to eliminate the potential for systematic frequency-dependent performance changes over the duration of the session. To encourage exploration of both nose poke holes, the mapping between the two nose poke sensors and associated stimulation frequencies was reversed electronically in mid-trial. The initial assignment of the two frequencies to the two sensors was also randomized among the trials. Trials were separated by 30 s inter-trial intervals and the LED indicator in the operant box was on during, but not in between the trials.

In house software controlled task variables and kept track of the rats’ choices and the number of rewards earned at reference and comparison frequencies during the sequence of trials. Reward titration curves were computed as the fraction of rewards harvested at the comparison frequency, as a function of the comparison frequency setting. Median filtering over a moving window of three consecutive frequencies was performed to remove outliers. Rats were required to generate an asymptotic titration curve on at least two sessions for inclusion in the data set, and to progress to the fMRI phase of the experiment. With the range of stimulus currents used, the titration curves typically saturated in a narrow range of frequencies near 137 Hz. Behavioral data processing was performed using Excel (Microsoft Corp, Redmond WA) and Matlab (Mathworks, Natick, MA); error bars denote the standard error of the mean across multiple measurements from individual animals or across multiple animals.

Magnetic resonance imaging

Animals that performed the task effectively (> 20 responses per minute; n = 10) were anesthetized with isoflurane and tracheotomized in preparation for functional magnetic resonance imaging (fMRI). Luer-fitted teflon endotracheal tubes (14 gage; McMaster-Carr, Santa Fe Springs, CA) were surgically inserted and fixed in place with dental floss (Walgreens, Cambridge, MA) and Vetbond (3M, St. Paul, MN). Ear canals were filled with toothpaste to avoid magnetic susceptibility-induced distortions in the images. Tracheotomized animals were paralyzed by intraperitoneal injection of a 1 mg/kg bolus of pancuronium bromide (Sigma-Aldrich, St. Louis, MO), followed by continuous 1 mg/kg/hr infusion throughout the course of subsequent experiments. Mechanical ventilation was performed using an Inspira ventilator from Harvard Apparatus (Holliston, MA), operated at 70 beats per minute and 5 mL per stroke. Heart rate and blood oxygenation were continuously monitored using an MRI-compatible noninvasive infrared sensor and pulse oximeter (Nonin Medical, Plymouth, MN), and isoflurane anesthetic concentration was initialized to 1% and adjusted within the range from 1.00–1.25% to keep physiological parameters within 10% of baseline values. Animals were wrapped in a heated circulating water blanket (Gaymar, Orchard Park, NY), and core body temperature was monitored by means of a rectal thermometer. They were then transferred to a positioning device (Ekam Imaging, Shrewsbury, MA) and inserted into an MRI scanner for imaging.

MRI scans were obtained using a 4.7 T 40 cm horizontal-bore magnet interfaced to a Bruker (Billerica, MA) Avance console and equipped with a 12 cm gradient set (26 G/cm). A transmit-only 10 cm inner diameter volume coil and a customized ~2 cm diameter receive-only surface coil (Ekam Imaging), both integrated into the animal positioning device, were used for excitation and detection. Pulse sequences were controlled through the Paravision 3.0 software (Bruker). For functional imaging with blood oxygenation level dependent (BOLD) contrast, a single-shot gradient echo echo planar imaging (EPI) pulse sequence was used with an echo time (TE) of 20 ms, recycle time (TR) of 2 s, slice thickness of 1 mm (14–16 slices centered over bregma), in-plane field of view 3.2 × 2.4 cm, and matrix size 64 × 48 points. Standard gradient echo BOLD was chosen because of its relative sensitivity compared with cerebral blood flow or spin echo-based measurement methods (Norris, 2012). High resolution anatomical images including electrode and cannula implantation sites were acquired using a conventional gradient echo pulse sequence with TE/TR = 15/2000 ms, slice thickness 1 mm, 3 × 3 cm FOV, and 256 × 256 matrix size. Diffusion tensor imaging (DTI) was performed after fMRI using a pulsed gradient spin echo sequence with EPI readout parameters matching those used for BOLD data acquisition. Data were obtained with 16-fold averaging and diffusion weighting parameter b = 0 and 1000 s/mm²; 15 gradient directions were applied. The pulsed gradient duration and diffusion time were 5 and 15 ms, respectively. An angiography dataset was acquired with a time-of-flight angiography pulse sequence with TE/TR of 5/30 ms.

Functional imaging experiments were performed in conjunction with MFB stimulation. Stimulating electrodes were connected to a constant current stimulator (World Precision Instruments) located outside of the scanner. Stimulation currents, pulse widths, and frequencies were individually matched for each animal to those used for behavioral experiments performed prior to imaging, and cable lengths were the same for imaging and behavior. To minimize potential radiofrequency artifacts, the stimulator cable was filtered with a 1.9 MHz low-pass filter (Mini-Circuits BLP-1.9, Brooklyn, NY) before entering the MRI enclosure. Stimulus pulses were monitored continuously on an oscilloscope (Tektronix TDS-210, Beaverton, OR) to verify consistency throughout scanning experiments. During fMRI data acquisition, stimuli were delivered in 40 s blocks of eight equally-spaced one-second long trains pulse trains, each at a specified frequency, separated by blocks of 60 s without stimulation. MFB stimuli of different frequency (10 frequencies) were delivered in pseudorandom order, and each sequence of stimuli was repeated three times for a total of 30 blocks of stimulation per fMRI experiment.

The average heart rate during imaging sessions was 428.7 beats per minute (bpm) with a standard deviation of 36.6 bpm. Heart rates were stable over the duration of each imaging experiment, with drift between start and end of a session ranging from −3.6% to 6.7%, and an average stimulus-induced change of 1.1%. Average heart rate changes to each stimulation frequency are shown in Supplementary Fig. S1, and remained under 2% in all cases. These stimulus-induced heart rate changes were small compared with changes of up to 5% reported in response to commonly used forepaw stimulation methods in rats (Liu et al., 2004) or changes of over 15% in human resting state studies (Chang et al., 2009; Shmueli et al., 2007).

Data analysis

Raw MRI data from all experiments were initially preprocessed using custom routines running in Matlab (Mathworks). All data were Fourier transformed with a spatial smoothing kernel of 1.5 voxels full-width at half maximum. EPI data were ghost-corrected using a published algorithm implemented in Matlab (Buonocore and Gao, 1997). Fractional anisotropy and directionality maps were computed from transformed DTI data using routines from the Matlab suite BEAR (University of Paris, France). Segmentation of brain from non-brain voxels was also performed in Matlab. Further preprocessing of functional imaging data was accomplished using the AFNI software package (National Institute of Mental Health, Bethesda, MD). Steps included slice timing correction, motion correction using a least squares rigid-body volume registration algorithm, voxel-wise intensity normalization, and segmentation of brain from non-brain voxels.

After motion correction, fMRI time series data from individual animals were coregistered onto a reference data set using a least squares affine alignment of fractional anisotropy maps (Supplementary Fig. S2a). Alignment of rodent fMRI datasets is more conventionally performed by registering each animal’s echo planar images to anatomical scans, and then registering between animals using the anatomical data. These are “information poor” procedures, since standard T₁ and T₂-weighted imaging do not offer many landmarks for coregistration in rodent brains. Use of the DTI maps for alignment guaranteed that spatial features throughout the brain were available to guide the registration. Because the DTI data were acquired using the same EPI readout parameters as the functional scans, DTI-based alignment also eliminated the need for within-animal alignment of functional with anatomical scans. Further, because subsequent region of interest (ROI)-based analyses were performed using regions defined by comparison of DTI and histological data, coregistration based on DTI guaranteed that the correspondence between individual animals’ EPI time series and anatomical regions remained relatively constant from animal to animal. Visual inspection of aligned fractional anisotropy data from multiple animals confirmed that individual features were effectively coregistered by our procedure. Time series data were smoothed with a Gaussian spatial kernel of 1 mm full width at half-maximum prior to statistical analysis.

Statistical analysis of preprocessed EPI time courses was performed using the general linear modeling (GLM) approach, implemented in AFNI (Cox, 1996). Regression vectors for calculation of activation maps were computed by convolving MFB stimulus times with a hemodynamic response model consisting of a single gamma function with Cohen parameters b = 6 and c = 0.8. Six motion correction parameters from each animal were included as nuisance regressors, along with a linear baseline term. Outlier scans detected by median deviation from time series trends in each data set were censored from the analysis. Stimulation frequency-independent average responses were determined by using a single regressor to model MFB stimulation at all delivered frequencies. Frequency-dependent responses were determined in a separate calculation, by using individual regressors for each stimulation frequency.

Response amplitudes (regression coefficients) determined for each voxel by the GLM analyses were used as input for further numerical analyses performed in Matlab. These included computation of ROI-averaged responses and correlation of fMRI responses with behavioral data. For ROI analyses, anatomically-defined regions were delineated by comparing DTI fractional anisotropy data and high-resolution anatomical scans with a standard brain atlas. Distinct landmarks recognizable by diffusion anisotropy, including fiber tracts such as the corpus callosum, anterior commissure, internal capsule, and cerebral peduncle, were used to relate an average of the coregistered DTI scans to individual images from a standard brain atlas (Paxinos and Watson, 1998). After assigning each DTI slice to the corresponding Horsley-Clarke A-P plane, atlas images were imported into Matlab and appropriately scaled to match the DTI images and high-resolution anatomical scans. ROIs were manually drawn over the DTI images, again making use of landmarks visible in each slice in either the DTI or anatomical image (Supplementary Fig. S2b). ROI names correspond to standard usage (Paxinos and Watson, 1998) in most cases, with the following additional notes: AmygAM and AmygPL include anteromedial and posterolateral amygdalar regions, respectively; CgCx includes cingulate cortex and adjacent limbic cortical regions; DpMe is broadly defined to include reticular and tegmental structures dorsal to VTA and SN and ventral to superior colliculus; Hippo includes standard divisions of the hippocampus; NAcC and NAcS correspond to core and shell regions of NAc; OCx includes ventral and lateral orbital cortex, plus immediately adjacent anterior insular cortical regions; Thal denotes the thalamus, without subdivisions; vSub includes ventral subiculum plus adjacent parasubicular and presubicular regions; VTA includes parabrachial, parainterfacial, and paranigral nuclei of the ventral tegmentum. The number of EPI voxels in each defined ROI is presented in Supplementary Table S1.

ROIs and statistical maps included all voxels in unbiased fashion, excepting areas identified as dominated by BOLD signals from large draining veins or signal drop out from electrode or cannula implants (Supplementary Fig. S3). The macrovascular compartments were taken to comprise areas that displayed strongly hysteretic responses to MFB stimulation (Lee et al., 1995). These were defined by comparing the amplitude for the final two stimulus trains and the initial two stimulation trains in each 8-train (40 s) stimulation block, as computed from a group of 10 animals. Voxels that showed 0.4% or greater average response to the last two stimulus trains vs. the first two were defined as predominantly macrovascular, and were excluded from statistical calculations and ROI averages. The voxel counts for each ROI in Supplementary Table S1 reflect this exclusion. For tests of the dependence of vascular ROI results on the precise definition of vascular regions, alternative vascular ROIs were delineated by thresholding a T₁-weighted angiography image, or by smoothing a map of internal edges detected using the Sobel method, implemented in Matlab with stringency parameters 10–25, applied to T₂^*-weighted anatomical maps (Supplementary Fig. S4). The first alternative approach proved most sensitive to major arteries, while the second method overlapped heavily with the hysteresis detection method, and was probably most sensitive to larger veins.

Unless otherwise noted throughout the paper, error bars reported for fMRI data denote the standard error of the mean values observed across multiple animals. Error estimates for ROI-averaged fMRI response amplitudes and stimulation frequency-dependent profiles were computed using jackknife resampling of the fMRI data from multiple animals, in conjunction with GLM analyses of the resampled group data. These analysis methods were implemented in AFNI and Matlab.

Assessment of reward tracking vs. frequency tracking by fMRI signals was performed by fitting voxel or ROI-averaged BOLD amplitudes at each MFB stimulation frequency to a GLM including a constant term and two regressors (Fig. 3a), one rising linearly from 0 to 1 in proportion to stimulation frequency (frequency tracking), and the other rising from 0 to 1 over the interval from 0 to 135 Hz and then remaining flat for frequencies over 135 Hz (reward tracking). The reward tracking regressor represents an idealized form of the reward titration curve of Fig. 2b, in which the rewarding value of a stimulus is assumed to be proportional to stimulation frequency up to the saturation point. All stimulation frequency-dependent data were smoothed by averaging over a moving window of three frequencies prior to GLM analysis or presentation in the figure panels. End points were not modified. Beta coefficients for the two regressors (i.e. estimated contributions of frequency and reward tracking models to the signal) were used to generate the red/green color coding of Fig. 3 or combined into the saturation index = β_R/(β_R + β_F) of Fig. 4a, where β_R and β_F are the reward and frequency tracking coefficients, respectively.

Fig. 3. Brain region-specific tracking of psychometric reward magnitude.

(a) Frequency tracking (green) and reward tracking (red) models used for regression analysis of stimulation frequency-dependent BOLD response amplitudes. The frequency tracking model predicts responses linearly proportional to the stimulation frequency, whereas the reward tracking model idealizes the saturation effect seen in the behavioral reward titration analysis, with an asymptotic saturation frequency equal to the experimentally observed value in Fig. 2b. (b) Voxel-level characterization of frequency vs. reward tracking by BOLD signal amplitudes in a group analysis of 8 rats. Amplitudes as a function of frequency were analyzed using a GLM incorporating frequency and reward tracking regressors from panel A. The amount of fMRI signal change (%) ascribed to frequency and reward tracking models is color coded such that pure frequency tracking appears green, pure reward tracking appears red, and an equal mixture appears yellow (color code shown bottom right). Maps are overlayed on a T₂-weighted anatomical scan, and ROIs and labels are shown as in Fig. 1c. (c) Close up of the two most pronounced foci of reward tracking near nucleus accumbens shell (NAcS, top) and adjacent to the amygdala (Amyg) near the stimulation electrode site (bottom). A standard histological map (Paxinos and Watson, 1998) is superimposed over the data. Voxels that showed statistically significantly (t-test p < 0.05) greater reward tracking than frequency tracking are outlined in blue; NAcS contains the largest cluster of significantly reward tracking voxels. (d) Plots of normalized mean BOLD amplitudes vs. stimulation frequency for several ROIs (mean ± s.e.m. shown for each in black, n = 8). Relative contributions of the two models to the BOLD signal from each ROI were expressed as coefficients β_R and β_F for reward and frequency tracking regressors, respectively. Corresponding p values for F-tests of the significance of each regressor’s contribution are noted. Pink dotted lines indicate the best fit regression curve in each case. Among the ROIs shown, NAcS and NAcC were the only regions that showed significant contribution of the reward tracking model, but not the frequency tracking model. Corresponding β_R values indicated that over 80% of the fMRI signal variation could be explained by the reward tracking model in both NAcC and NAcS.

Fig. 2. Reward titration analysis and behavioral results.

(a) Diagram of the apparatus for measuring reward titration curves. Rats implanted with an MFB stimulation electrode and connected to a constant current stimulator (red) were placed into an operant chamber with two nose poke holes. One hole was associated with a stimulation reward delivered at a fixed reference frequency (blue) while the second hole was associated with a test stimulation frequency that varied from trial to trial (green). Timing of the trials and stimuli is automated by custom software running on a computer. (b) The percentage of test operant actions is plotted as a function of test frequency for a range of frequencies, offered in the two-choice test against a reference frequency of 150 Hz. Stimulation frequencies below 134 Hz are increasingly rewarding and are chosen with increasing probability as they approach the reference. Frequencies above 134 Hz are virtually indistinguishable from 150 Hz stimulation, giving rise to the flat response profile at high frequency that characterizes the reward saturation phenomenon. The graph shows mean and s.e.m. for 8 rats.

Fig. 4. Vascular contributions do not explain saturating fMRI profiles.

(a) BOLD amplitudes observed at the maximal stimulation frequency (horizontal axis) were plotted vs. an index of response saturation, equal to β_R/(β_R + β_F), where β_R and β_F are regression coefficients for reward and frequency tracking models (see text and Fig. 3). Mean and s.e.m. values are shown across animals (n = 8) for each ROI, and for a vascular compartment (black). Dotted line shows best linear fit to the data (R = 0.17, p = 0.47). Inset shows correspondence between BOLD amplitudes to 231 Hz stimulation and saturation index for individual voxels, with linear regression fits in three ROIs: NAcS (red; R = 0.15, p = 0.39), NAcC (orange; R = 0.55, p = 0.015), and AmygAM (light green; R = 0.21, p = 0.32). NAcC showed statistically significant anti-correlation, and other regions were uncorrelated, failing to support the hypothesis that reward tracking BOLD responses result from saturation of neurovascular coupling. (b) Representative macrovascular regions identified on the basis of hysteretic responses to MFB stimulation (see Methods). Sagittal sinus and vessels near the third ventricle shown in slice −2.8 mm from bregma. (c) Macrovascular BOLD response amplitudes as a function of MFB stimulation frequency. Regression analysis with reward tracking and frequency tracking models are shown as in Fig. 3d, with corresponding β and F-test p values noted. The frequency tracking model accounted for the majority of variance.

Amperometric measurement of dopamine

Four rats were implanted with stimulation electrodes, shaped to perform operant responses, and trained for 3–5 days according to methods described above. Prior to amperometry, animals were anesthesized with 2% isofluorane, warmed on a heating pad and placed in a steoreotax. A carbon fiber recording electrode was implanted in NAc, 1.6 mm anterior to bregma, 1.5 mm left of midline and 8.4 mm below the skull surface. An Ag/AgCl reference electrode was implanted at identical coordinates contralateral to the carbon fiber electrode. The recording and reference electrode were connected to a Picostat and e-corder system (eDAQ Inc., Colorado Springs, CO), which applied a fixed potential of 0.8 V to the recording electrode and continuously monitored the oxidation current with 10 kHz sampling. Once a stable baseline was achieved, MFB stimulation was delivered as above in blocks of eight 1 s pulse trains (0.2 ms pulse width), with 4 s inter-train interval and 60 s separation between stimulation blocks. Stimuli were applied with pulse frequencies of 44–231 (n = 1) or 44–400 Hz (n = 3), delivered twice each in pseudo-random order. The paradigm was repeated with a control applied potential of 0.1 V to verify specificity of the measurements to the oxidation potential of dopamine. Following recording, carbon fiber electrodes were calibrated in a custom-made flow cell, using dopamine concentrations of 0–10 μM in the presence of 600 μM ascorbic acid. Calibration constants of 0.6 ± 0.2 nA/μM dopamine (mean ± s.d.) were obtained.

Targeted pharmacological inactivation

Six rats were implanted with monopolar electrodes in the MFB and cannulas over the ipsilateral NAc. The surgical procedures and coordinates were the same as those described above. The rats were then trained to perform reward-seeking behavior also as described above, except that the number of comparison frequencies was reduced to four (50, 90, 150, and 202 Hz) to limit the length of the experiment and thus the total infusion volume. The day after the rats achieved a stable titration curve the inactivation experiment was performed. Rats were first anesthetized with 1% isoflurane to facilitate the placement of an internal cannula into the NAc. Injection cannulae were inserted into NAc through the preimplanted guide cannulae, such that the injection tip position was −1.7 mm anterior and 0.8 mm lateral to bregma and 7.5 mm below the skull. Cannulae were then fixed in place with dental cement to prevent dislodgement during the behavioral task. Animals were allowed to recover in the behavioral box, and infusion began once they started to move around. The first (dummy) trial commenced five minutes later, followed by the four test trials delivered in pseudo-random order. All trials were five minutes long. Infusion of 2% lidocaine continued throughout the experiment at a rate of 0.1 μL / min. The total infusion volume thus ranged from 1 μL (at the start of trial 1) to 3 μL (at the end of trial 4). This continuous infusion model was utilized to prevent any lidocaine washout during the experiment. The next day the normal behavioral task was repeated to verify that any observed results were not the result of cannula placement or tissue damage from the infusion. These procedures were also applied to an additional five rats with infusion cannulae implanted over ipsilateral VTA rather than NAcS. In each ease, the electrode and cannula placements were verified for each animal with anatomical MRI scans on a 7 T Bruker system.

RESULTS

MFB stimulation activates mesolimbic and nigrostriatal pathways

We first used fMRI to study the spread of MFB stimulation-evoked activity from the electrode site in behaviorally characterized rats. Rats were chronically implanted with stimulating electrodes in the MFB at the level of the lateral hypothalamus (LH), and were trained in an operant task to perform nose-pokes in response to rewarding stimuli of varying frequency (44–231 Hz). Animals that performed the task effectively (n = 10) were investigated using blood oxygenation-level dependent (BOLD) fMRI in a 4.7 T scanner. Imaging experiments were performed under 1–1.25% isoflurane, and stimuli delivered in the scanner matched those used for behavioral experiments. Representative time courses from individual regions displayed modulation as a function of varying microstimulation frequency (Fig. 1a) and temporal microstructure reflecting individual stimulus pulse trains (Fig. 1b), indicating stimulus specificity of the observed MRI signal changes. Mean time courses averaged in unbiased fashion over anatomically-defined regions in multiple animals showed similar modulation (Supplementary Fig. S5).

Fig. 1. BOLD fMRI responses to MFB reward stimulation. (a).

Representative blood oxygenation level dependent (BOLD) fMRI response to electrical stimulation of the medial forebrain bundle (MFB), averaged over all voxels in the lateral hypothalamus (LH) ipsilateral to the stimulation electrode in a single rat. Ten stimulation frequencies (green labels, in Hz) were presented in pseudo-random order and the stimulation sequence was repeated three times over the duration of the scan session. Each green rectangle corresponds to one stimulation block. (b) Average LH response to a 40 s block of eight MFB stimulation trains delivered at 231 Hz, showing peaks in the response profile following each 1 s train (vertical red lines). The gray shading indicates standard error of the mean (s.e.m.) over three stimulus presentations in a single animal, at time points with respect to the stimulus onset at t = 0 seconds. (c) Map of average fMRI responses to MFB stimulation over stimulation frequencies from 44–231 Hz in a group of ten animals. Data are displayed as an array of 1 mm slices from rostral to caudal, beginning with the most rostral slice at the lower left (coordinates with respect to bregma displayed to the top left of each slice, in yellow). Background images are T₂-weighted anatomical images. Color overlays in red-yellow depict the BOLD amplitude as % signal change, according to the color bar (lower right). Colored outlines depict boundaries of 21 regions of interest (ROIs); only ROIs contralateral to the electrode (white shaded bar, −1.8 from bregma) are drawn. Color-coded ROI labels at bottom correspond to the ROI outlines. Data were thresholded for statistically significant activation with partial Bonferroni correction (p ≤ 0.0001). (d) ROI-averaged mean BOLD signal modulations elicited by 44–231 Hz MFB stimulation, arranged from maximum to minimum amplitude for ROIs ipsilateral to the stimulation electrode. Contralateral response amplitudes are shown in the inset. Error bars denote s.e.m. over animals (n = 10). Signal change calculations included all voxels in each ROI, excepting regions of signal drop out due to implants and areas dominated by macrovascular responses, which were excluded from ROI averages.

A map of the average BOLD signal amplitudes observed in response to MFB stimulation is presented in Fig. 1c. Response amplitudes averaged over 21 anatomically-defined ROIs per hemisphere were computed and are presented in Fig. 1d, and corresponding unmasked images and statistical maps are shown in Supplementary Fig. S6. Substantial responses were measured in cortical and subcortical brain areas, in total accounting for over 50% of brain voxels. Response amplitudes averaged over the full range of stimulation frequencies (Fig. 1d) were lower on average than responses to some of the higher individual frequencies, but similar relative amplitudes were observed among the ROIs (Supplementary Fig. S7a). Response amplitudes varied somewhat among animals, but few outliers were observed (Supplementary Fig. S7b). Response magnitudes were also similar between groups of animals stimulated with higher vs. lower currents (see Methods and Supplementary Fig. S8). The regions most strongly activated by ipsilateral MFB stimulation included structures commonly associated with reward, such as LH (0.83 ± 0.22% signal change), the nucleus accumbens shell (NAcS, 0.65 ± 0.16%), preoptic area (PO, 1.45 ± 0.54%), olfactory tubercle (Tu, 0.68 ± 0.17%), and the VTA (0.89 ± 0.20%). Motor control areas were also modulated, with pronounced BOLD signal observed in the substantia nigra (SN, 0.77 ± 0.14%), and robust activation also in the thalamus (Thal, 0.48 ± 0.08%) and caudate-putamen (CPu, 0.36 ± 0.06%). Regions contralateral to the stimulation electrode were activated by the stimulus as well (Fig. 1d inset). The two most pronounced foci of BOLD signal on the contralateral side were observed around Tu (0.90 ± 0.45%) and VTA (0.72 ± 0.21%), suggesting that one or both of these areas might receive relatively direct input via structures activated by MFB stimulation on the side ipsilateral to the electrode.

BOLD responses in specific brain areas track psychometric reward measurements

Brain regions showing fMRI-detectable responses to MFB stimulation may or may not participate in mediating rewarding effects of the stimulus. Moreover, among areas that are required for brain stimulation reward, only a subset might be required for computing or representing reward magnitudes. We hypothesized that the neural populations that encode results of behaviorally-relevant reward magnitude evaluation, if they exist, would show BOLD fMRI signal change amplitudes most closely correlated with a psychometric indicator of subjective reward magnitude. Neural activity induced by the stimulus but prior to or uninvolved with reward integration would be more likely simply to follow the intensity of the stimulus itself. Similarly, neural activity associated with fibers adventitiously activated by the stimulus but not involved with rewarding effects might also be more likely to track the stimulus intensity, rather than its apparent reward value.

A psychophysical measurement of reward magnitude can be obtained by determining operant response rates as a function of MFB stimulation frequency (Gallistel and Leon, 1991). Such “reward titration curves” have been shown to display a characteristic asymptotic shape, in which response rates rise with increasing frequency up to a “saturation point,” above which larger stimulation frequencies produce roughly constant responses (Mark and Gallistel, 1993; Simmons and Gallistel, 1994). The asymptotic shape of the reward titration curve as a function of MFB stimulation frequency is thought to arise from distributed network properties, rather than from frequency following failure at or near the site of stimulation (Solomon et al., 2015). According to our hypothesis, a brain region or voxel that contains neurons most closely associated with an animal’s subjective assessment of reward magnitude in the MFB stimulation paradigm would therefore show BOLD activation amplitudes that correlate with the reward saturation curve, as opposed to the stimulus intensity per se.

Emulating earlier studies (Gallistel and Leon, 1991; Mark and Gallistel, 1993; Simmons and Gallistel, 1994), we measured reward titration curves using a matching task in which animals compared two MFB stimuli of differing intensity, each associated with a nose-poke sensor (Fig. 2a). Across multiple trials, one stimulus was fixed at or above the reward saturation frequency for each animal, and the second stimulus was varied over a frequency range around the saturation point. Measuring reward titration curves in this way controlled for factors that could degrade operant performance without specific dependence on stimulation frequency. Eight of the ten animals included in the fMRI analysis of Fig. 1 reached stable performance in this titration procedure for two consecutive daily sessions prior to scanning. Group average behavioral data from these animals are presented in Fig. 2b. The reward saturation frequency, defined as the lowest MFB stimulation frequency that elicited ≥ 45% of the operant responses, compared with a reference operant associated with 150 Hz stimulation, averaged 137 ± 3 Hz across the group; all individual animals in the group exhibited reward titration curves that reached saturation at 134 or 161 Hz and averaged between 47–59% for MFB stimulation frequencies above 134 Hz.

BOLD response amplitudes were measured from these same eight animals as a function of MFB stimulation frequency for the ten frequencies (44–231 Hz) used in behavioral testing. Curves of BOLD amplitude vs. frequency were then subjected to a regression analysis to determine the extent to which fMRI signals paralleled the behavioral results of Fig. 2b. The regression analysis involved fitting the fMRI data from each voxel to a linear combination of two idealized model functions (Fig. 3a), one simulating the saturating behavioral reward titration curve (“reward tracking”) and the second varying linearly with stimulation frequency (“frequency tracking”), plus a constant. Reward tracking and frequency tracking regression coefficients, β_R and β_F, respectively, indicated the fraction of fMRI response amplitudes that could be explained by each of the two models. A map of β_R (red) and β_F (green) values over the brain (Fig. 3b and Supplementary Fig. S9a), scaled by response amplitudes, reveals two pronounced foci of reward tracking signal in NAcS and near the electrode site, adjacent to the amygdala in an area approximately corresponding to the substantia innominata (SI). In these two regions, BOLD responses of up to approximately 2% were fully accounted for by the reward tracking model. Individual voxels showing significantly more reward tracking than frequency tracking character were identified by a t-test of β values obtained over multiple animals. Among regions with strong reward-tracking character, the largest focus of voxels that reached uncorrected statistical significance (p < 0.05) in the t-test was found in NAcS (Fig. 3c and Supplementary Fig. S9b), indicating that BOLD signal in this region most strongly reflects the psychophysical variable required to explain the reward titration behavior of Fig. 2b.

Reward vs. frequency tracking behavior was also assessed for anatomically-defined ROIs. Mean responses to each stimulus frequency are plotted in Fig. 3d for several regions ipsilateral to the electrode placement site; analysis of additional regions is summarized in Table 1. Data from brain areas including NAcS, NAcC, and AmygAM showed responses that appeared to correspond closely to the behavioral data, increasing for low frequencies and then reaching a plateau around the saturation frequency of 137 Hz. Other areas, such as the VTA and Hippo showed monotonically increasing responses, approximately proportional to the MFB stimulation frequency. Each of these ROI-specific curves was normalized and then fit to a regression model incorporating reward and frequency-tracking vectors. Across ROIs, the fraction of variance accounted for by the reward tracking models was anticorrelated with the variance accounted for by the frequency tracking model (R ≤ −0.82, p < 10⁻⁴, n = 21 for both hemispheres), indicating that the two models tend to be mutually exclusive.

Table 1.

Reward vs. frequency tracking in 42 ROIs

ROI*	ipsilateral				contralateral
	βR	p	βF	p	βR	p	βF	p
AmygAM	0.75	0.0030	0.45	0.033	0.16	0.57	0.91	0.011
AmygPL	1.14	0.0016	0.24	0.33	1.48	0.012	0.16	0.72
CPu	0.16	0.57	0.82	0.019	0.30	0.078	0.61	0.0038
CgCx	0.13	0.65	0.60	0.074	0.34	0.054	0.40	0.030
DpMe	0.82	0.0004	0.63	0.0017	0.76	0.0007	0.61	0.0024
GP	0.22	0.42	0.75	0.022	0.69	0.0015	0.40	0.023
Hippo	0.40	0.011	0.89	0.0001	0.89	< 10−4	0.44	0.0006
LH	0.64	0.0001	0.55	0.0001	−0.86	0.14	1.61	0.017
LS	0.25	0.29	0.82	0.0075	0.01	0.98	1.14	0.013
NAcC	0.82	0.023	0.15	0.60	−0.02	0.96	0.97	0.032
NAcS	1.10	0.0003	0.03	0.86	0.89	0.029	−0.02	0.94
OCx	−0.01	0.98	1.23	0.014	0.20	0.39	0.72	0.012
PO	0.53	0.0001	0.69	< 10−4	0.68	0.0041	0.76	0.0023
PiCx	0.05	0.84	1.04	0.0024	0.40	0.016	0.69	0.0009
SCx	0.50	0.10	0.80	0.020	0.59	< 10−4	0.56	< 10−4
SN	0.68	0.0093	0.59	0.018	0.64	0.0009	0.36	0.019
Thal	0.42	0.0009	0.54	0.0002	0.59	< 10−4	0.39	0.0003
Tu	0.58	0.0008	0.46	0.0030	0.13	0.078	0.78	< 10−4
VP	0.39	0.032	0.70	0.0020	0.50	0.0050	0.66	0.0011
vSub	−0.03	0.95	1.80	0.015	0.76	0.17	0.65	0.23
VTA	0.49	0.0008	0.71	0.0001	0.52	0.0016	0.69	0.0003

^*Region of interest (ROI) definitions and data analysis were performed as described in the main text. β_R and β_F denote relative contributions of reward tracking and frequency tracking models, respectively, to BOLD amplitudes recorded as a function of MFB stimulation frequency. Data for ROIs both ipsilateral and contralateral to the MFB electrode are provided. See text and Fig. 3 for further details. The number of voxels in each ROI is listed in Table S1.

The brain areas with average BOLD signal most strongly correlated with the reward magnitude tracking model (β_R ~ 1 and β_F ~ 0) were the NAcS, NAcC, AmygPL, and AmygAM. Data from NAcS, NAcC, and AmygPL only could be explained solely using the reward tracking regressor (F-test p ≤ 0.02 for reward tracking, p ≥ 0.3 for frequency tracking). Of these three regions, NAcS displayed the greatest tendency for reward tracking, with β_R = 1.1 and β_F = 0.0, as well as the greatest BOLD response to maximal MFB stimulation, 1.02 ± 0.22%, vs. 0.89 ± 0.16% for NAcC, and a much lower 0.28 ± 0.10% signal change for AmygPL. Interestingly, contralateral NAcS was one of the most highly reward tracking structures in the hemisphere opposite the microstimulation electrode, as was contralateral AmygPL (Table 1), but the latter again with much lower amplitude. Results from the behavior vs. frequency tracking model analysis are probably determined by differences in the relative amplitudes of responses to stimulation frequencies above the saturation frequency of 150 Hz. In keeping with this idea, a straightforward evaluation of the mean slope of response amplitudes as a function of stimulation frequency above 150 Hz confirmed that NAcC and NAcS displayed the lowest, most plateau-like slopes (Supplementary Fig. S10). These results together indicate that NAc activity, particularly in the shell, most closely tracks the psychometric reward value recorded in the behavioral test, and that NAcS responses are the most reliable indicator of MFB stimulation reward value in the brain.

Stimulus tracking responses do not result from nonlinear neurovascular coupling

Reward magnitude tracking voxels in NAcS and elsewhere exhibited BOLD response amplitudes that reached asymptotic maximal signal changes at MFB stimulation frequencies near the behaviorally-determined reward saturation frequency. Although a likely interpretation of the results is that these brain regions contain neural populations that specifically encode reward magnitudes, a possible confound could arise from nonlinear behavior of the BOLD signal itself (Logothetis, 2002; Logothetis et al., 2001)—perhaps apparent saturation of the fMRI responses with increasing stimulation frequency results from limited dynamic range of the neurovascular coupling relationship itself, as opposed to the underlying neuronal activity. If nonlinear neurovascular coupling were the explanation for apparent reward magnitude tracking fMRI signals, the saturation effect would likely be most prominent in voxels with the largest BOLD response magnitudes.

To test this explanation, we compared the peak fMRI signal changes in each ROI with an index of reward magnitude tracking given by β_R/(β_R + β_F) (Fig. 4a); index values close to one indicate a greater relative contribution of the saturating reward tracking model to BOLD response variation as a function of stimulation frequency. We found little correlation between β_R/(β_R + β_F) and maximal signal changes (R = 0.17, p = 0.47), and even less correlation between maximal signal changes and β_R values (R = 0.04, p = 0.85). We also compared the peak BOLD signal changes with β_R/(β_R + β_F) for individual voxels in three of the most strongly reward tracking ROIs, NAcS, NAcC, and AmygAM (inset). AmygAM (R = 0.21, p = 0.32) and NAcS (R = 0.15, p = 0.39) showed no significant correlation, whereas NAcC showed a modest anticorrelation (R = −0.55, p = 0.015) that cannot easily be explained by the hypothesis that nonlinearity of regional BOLD responses favors reward-tracking.

As a further test, we examined whether MRI signal changes observed in an identified macrovascular ROI (Fig. 4b) exhibited apparent reward magnitude tracking as a function of MFB stimulation frequency (Fig. 4c). The macrovascular ROI was defined by the observation of hysteretic responses to stimulation (Lee et al., 1995), and was consistent with known veinous distributions in the rat brain (Paxinos, 2004) and hypointense regions in our anatomical scans (Supplementary Fig. S4), rather than with brain areas most susceptible to global physiological changes in rodents (Schroeter et al., 2014). Stimulus-associated signal changes observed in the macrovascular regions were also lateralized toward the brain hemisphere that received direct MFB electrode stimulation (Supplementary Fig. S6), and probably therefore reflect drainage of deoxygenated blood from stimulated tissue. We found that the majority of the macrovascular signal variation was explained by the frequency tracking model (β_F = 0.63 vs. β_R = 0.50). Although this ROI showed one of the largest average BOLD signal amplitudes, it did not display a particularly high saturation index (black data point in Fig. 4a). Similar results were obtained under alternative definitions of the vascular ROI (Supplementary Fig. S4). These findings again suggest that reward-tracking asymptotic behavior of some of the regional BOLD signal amplitudes is not a general consequence of nonlinear neurovascular coupling or brain-wide hemodynamic artifacts.

Neither VTA activity nor ventral striatal dopamine release tracks reward value

Given the prominent role of VTA input to NAcS, it was notable that VTA neural activity measured by BOLD fMRI failed to correspond closely to psychometrically determined reward values in the fMRI experiments. To further probe the correspondence between VTA activity and observed responses, we measured VTA-dependent dopamine release amplitudes in NAcS using fixed potential amperometric detection in four rats trained to perform operant responses to MFB stimulation. These experiments applied stimulation conditions similar to those used in our behavioral and imaging experiments in rats anesthetized without paralytic by 2% isoflurane. All animals showed monotonically increasing electrochemical responses to MFB stimulation with a current of 0.35 mA and increasing frequency ranging from 44–231 Hz. In three animals tested over an extended stimulation frequency range up to 400 Hz, continually increasing responses were observed for frequencies up to 315 ± 32 Hz (mean ± s.d.; Fig. 5). This is significantly higher than saturation frequencies of 158 ± 35 Hz observed for each of six rats trained with stimulation currents (0.3–0.4 mA) comparable to those used in the electrochemistry experiments (t-test p = 0.0003). The dopamine amperometry results are therefore similar to fMRI responses in VTA, which also ed reward stimulation frequency but not the behaviorally measured subjective reward magnitude. These results indicate that the reward-tracking fMRI signals observed in ventral striatum do not arise solely from dopaminergic VTA input.

Fig. 5. NAc dopamine signals saturate only at very high frequency.

(a) Dopamine oxidation detected by fixed potential (0.8 V) amperometry during MFB stimulation in a representative animal under conditions similar to the fMRI experiments. Each trace shows responses to a series of eight pulse trains delivered with the stimulation frequency indicated. (b) Normalized average amperometric responses (n = 3) measured over a range of frequencies including those used for behavioral and MRI experiments. Saturation is observed only at frequencies of 315 ± 32 Hz (mean ± s.d.).

Targeted silencing of NAcS distorts the reward titration curve

The correspondence between psychometrically determined reward magnitudes and BOLD signal profiles in the NAcS suggests that neural population activity in NAcS may play a role in calculating or storing information about reward value during behavior. To test this hypothesis, we performed transient local inactivation of NAcS during operant behavior. Six additional rats were each implanted with a guide cannula placed above the medial NAcS (Fig. 6a), ipsilateral to an MFB stimulation electrode. These animals were trained on a reduced version of the matching task of Fig. 2, in which their preference between four randomly presented frequencies vs. 150 Hz stimulation was measured in the double nose-poke task. Reward titration curves were recorded after establishment of stable performance. On the subsequent day, the curves were remeasured during infusion of 2% lidocaine via an injection cannula lowered into NAcS. Absolute operant response rates were barely affected by the treatment, with averages of 33 ± 2 and 35 ± 2 rewards/minute harvested prior to and during lidocaine infusion, respectively (t-test p = 0.40). This indicates that the shift in the reward titration curve produced by lidocaine infusion into NAc cannot be explained simply as the result of a decrease in the rewarding value of MFB stimulation delivered at each frequency.

Fig. 6. Targeted inactivation of NAcS perturbs reward titration behavior.

(a) Cannulae were implanted in the medial NAcS of rats, ipsilateral to MFB stimulation electrodes; correct placement was verified by MRI at 7 T. Arrowhead denotes cannula tip in a representative animal. Scale bar = 5 mm. (b) Reward titration curves were measured using methods analogous to those of Fig. 2, but with a reference stimulation frequency of 150 Hz and four comparison frequencies of 50, 90, 150, and 202 Hz. Data were obtained from six animals that had reached stable performance on the task, and graphs represent the relative number of rewards harvested from the nosepoke hole associated with the titrated stimulation frequency the day before treatment (left), during infusion of 0.1 μL/min of 2% lidocaine into NAcS (middle), and on the day after the perturbation (right). Black curves denote measured values with error bars over six animals; dashed magenta lines are best fit from a regression model including linear and saturating components as in Fig. 3a. (c) Average regression coefficients for linear and saturating models (β_L and β_R, respectively) fit to individual animal reward titration curves obtained before, during, and after NAcS inactivation, showing that inactivation abolishes the saturating profile with β_R > β_L (p ≤ 0.005) observed both before and after lidocaine treatment.

Reward titration curves obtained in the presence of NAcS-targeted inhibition showed a notable change with respect to baseline behavior however, appearing more linear and less saturating when compared with pre-lidocaine reward titration curves (Fig. 6b). When the same animals were remeasured again after washout of the lidocaine, the curves regained their saturating profile, ruling out nonspecific effects of cannula implantation and fluid infusion on the behavioral results. The changes in reward titration curve could be quantified by applying the regression analysis of Fig. 3 to the behavioral data (Fig. 6c). β_R and β_F values before lidocaine treatment were 1.3 ± 0.1 and 0.04 ± 0.19, respectively (significant with p = 0.00015). β_R and β_F values during inhibition were 0.5 ± 0.4 and 0.7 ± 0.4 (not significant with p = 0.8), but after recovery the significant difference (p = 0.005) was restored with β_R = 1.7 ± 0.4 and β_F = −0.5 ± 0.5. The fact that NAcS-targeted inactivation perturbed reward titration behavior without attenuating operant response rates per se is consistent with the fMRI analysis in indicating the importance of NAc for evaluation of subjective reward magnitudes.

Reward titration curves were also obtained in the presence of analogous targeted inhibition of VTA (Supplementary Fig. S11). Infusion of 2% lidocaine into VTA also perturbed reward titration behavior. In this case, β_R and β_F values before lidocaine treatment were 1.3 ± 0.3 and 0.02 ± 0.29, respectively (significant with p = 0.014, n = 5). β_R and β_F values during VTA inhibition were 0.13 ± 0.56 and 1.0 ± 0.5 (not significant with p = 0.26). Return of the behavior to pre-treatment values following the procedure was incomplete, indicating the possibility that VTA cannula placement or infusion caused minor damage. In contrast to the slight increase in operant response rate associated with lidocaine infusion into NAc, targeted infusion into VTA produced a modest decrease in the number of operant responses per minute (Supplementary Fig. S12), from 35.0 ± 0.6 to 30.5 ± 2.2 (t-test p = 0.07). These results are consistent with the well-established importance of VTA as a component of reward circuitry and source of input to NAc, as well as with evidence that VTA activity is less specifically involved in reward magnitude processing.

DISCUSSION

Using a combination of fMRI, electrical microstimulation, and quantitative behavioral measurements in rats, we have demonstrated that the psychometric magnitude of brain stimulation rewards delivered to LH is most consistently tracked by hemodynamic population activity signals in the shell of the nucleus accumbens. Of 42 other ROIs surveyed across the brain in our study, only a subregion of the amygdala exhibits reward-correlated behavior comparable to NAc. Although the nonlinear nature of neurovascular coupling complicates interpretation of fMRI findings in neurophysiological terms, reward magnitude tracking was not observed in macrovascular compartments and did not seem to arise from saturation of ROI-specific hemodynamic responses. In addition, we found that targeted inactivation in NAc, ipsilateral to rewarding stimulation, correspondingly distorts the reward titration curve in awake, behaving rats without substantially altering their overall willingness to work for reward. These results together suggest therefore that a behaviorally-relevant quantitative measure of brain stimulation reward magnitude is indeed encoded by neural population activity in NAc. Because of the evolutionary conservation of neuroanatomical substrates thought to mediate reward processing, this finding is relevant to the understanding of reward integration and mesolimbic function in both animals and humans.

Our experiments made use of a well-characterized artificial reward, electrical stimulation of the MFB (Olds and Milner, 1954), in place of natural primary rewards such as juice or sucrose solutions. Despite its unphysiological origin, the reward sensation produced by MFB stimulation elicits naturalistic behavioral responses (Olds and Milner, 1954), and can both substitute for (Green and Rachlin, 1991) or compete with (Conover and Shizgal, 1994) natural rewards. Another unphysiological aspect of our experiments was the fact that fMRI brain activity mapping was performed in lightly anesthetized rats, as opposed to the awake, behaving condition used for psychophysical measurements to determine MFB stimulation reward magnitude. This approach is justified in part by previous results showing that fMRI measures of functional connectivity in rats are relatively unperturbed by the isoflurane level used (Wang et al., 2011), and also by the fact that similar levels of MFB stimulation-evoked dopamine release have been observed in anesthetized vs. awake animals (Tepper et al., 1991); dopamine release also appears relatively independent of isoflurane levels up to 3% (Brodnik and Espana, 2015). The comparison of experimental results across differing physiological conditions is also validated by the fact that NAcS reward tracking in fMRI was shown to be consistent with the targeted inactivation studies we subsequently performed in awake animals.

Some of the conclusions of this study are based on the assumption that the BOLD fMRI amplitudes are an approximately linear measure of neural population activity levels. BOLD responses have been shown to arise primarily from synaptic activity (Logothetis et al., 2001; Rauch et al., 2008), which induces functional hyperemia through parallel chemical signaling pathways involving both neurons and glia (Iadecola and Nedergaard, 2007). The roughly linear relationship between local synaptic activity and hemodynamic responses is known to be violated under conditions of prolonged or intense activity (Logothetis, 2002), so we performed internal controls to assess whether the apparent psychometric reward tracking saturation of BOLD responses we observed resulted from nonlinear neurovascular coupling (Fig. 4 and Supplementary Fig. S4). We found that voxels and ROIs with larger BOLD responses did not systematically display more saturating behavior than those with lower BOLD responses; we also found that macrovascular fMRI signals did not show saturating responses with increasing MFB stimulation frequency. Further, the frequency at which BOLD responses approached their asymptotic value matched the saturation frequency observed in behavioral reward titration experiments (~137 Hz), and took place at stimulation frequencies that in a parametric study of cortical fMRI activation did not induce saturation of hemodynamic responses (Logothetis et al., 2010). Thus, while we cannot rule out that nonlinearities in neurovascular coupling may have influenced the stimulus intensity-dependent signal amplitudes we report, they do not explain the observed correlations with psychometric reward measurements. In the future, electrophysiological measurements could be made to further characterize the relationship between fMRI signals we recorded and underlying neural activity patterns at the cellular level.

It was a hypothesis of our study that foci of neural population activity correlated with psychometric reward titration curves could be identified from fMRI data; we in fact discovered such foci in NAc and, less robustly, near AmygAM (red in Fig. 3). In addition to these hot spots of reward magnitude tracking, we found multiple areas where BOLD signal correlated roughly linearly with rewarding stimulation frequency (green in Fig. 3). Neural population activity averages in these frequency tracking regions cannot predominantly reflect subjective assessments of reward magnitude because they do not exhibit the saturating profile required to explain behavioral performance in the matching tests we performed. Frequency tracking areas probably include both “first stage” reward-related fibers (Gallistel et al., 1981) that transmit MFB stimuli to sites of integration and behavioral control, and also areas not naturally involved in reward processing. While fractional reward magnitude tracking behavior was observed in several ROIs (indicated by β_R values between 0 and 1; Table 1), these regions were modulated to a lesser extent or less reliably than hemodynamic signals in NAcS. Because BOLD responses reflect gross population-averaged activity, it likely that individual cells both outside and inside NAcS respond to MFB stimuli with characteristics that differ substantially from the mesoscale fMRI readouts. It is possible that cells in distal structures contribute to saturating BOLD response profiles in NAc (as suggested also by the effect of VTA inhibition on reward titration behavior), and it is also possible the balance of reward vs. frequency tracking in some structures might shift if different rewarding stimuli were used. Assuming approximate linearity of BOLD signaling, our results do however rule out the possibility that a single well-resolved neural population outside NAc robustly signals reward magnitudes under all conditions, since we only observed strong reward magnitude tracking activity in NAc.

Although the mechanisms by which reward tracking activity is established in NAcS may be complex, two pieces of evidence suggest that factors other than VTA input and striatal dopamine release may be involved. First, caudal ROIs including VTA did not show strong reward tracking BOLD signal in the stimulus titration analysis. Second, electrochemistry experiments dissociated dopamine release from fMRI signals in NAc and from behavioral performance in the reward titration tests, a result consistent with previously reported neurochemical measurements (Cossette et al., 2015; Garris et al., 1999; Kilpatrick et al., 2000; Miliaressis et al., 1991). Thus, despite evidence that stimulation of dopamine neurons is sufficient for behavioral reinforcement (Kim et al., 2012; Witten et al., 2011), and that VTA inactivation perturbs reward titration behavior (Supplementary Fig. S11), our results suggest that NAc activity ipsilateral to MFB stimulation reward relies at least to some extent on non-dopaminergic inputs to NAc (Britt et al., 2012). Further mechanistic analysis of the population-level responses we observe, in terms of contributions from specific neural projections or neurotransmitter-dependent systems, must be performed using more targeted and invasive techniques.

These results are relevant to the interpretation of BOLD responses in human fMRI studies of the reward system. Earlier reports have associated NAc BOLD signals with dopamine release and argued that these signals correspond to activity of VTA dopaminergic neurons as measured in nonhuman primates (McClure et al., 2003; O’Doherty et al., 2003). Two of our results—the fact that NAc and VTA fMRI signals do not similarly correspond to reward magnitudes, and the finding that amperometric dopamine recordings fail to parallel fMRI activation measured under similar conditions—suggest that NAc BOLD signals are not a faithful indicator of dopaminergic signaling. Also relevant to human fMRI findings is our failure to discern reward magnitude tracking in OCx, an area that includes orbitofrontal regions shown to signal reward values in associative tasks (Murray et al., 2007), and to which human studies have attributed reward magnitude sensitivity (Kringelbach, 2005). Relatively weak responses we observed in OCx and in cortical areas more generally suggest that the MFB stimulation paradigm most effectively engages evolutionarily primitive components of reward processing circuitry, and that these circuit elements are also most directly involved in reward magnitude computations.

Our analysis of reward magnitude-related neural population activity in rats supports several existing lines of evidence that point to a particularly important role for NAc neuronal activity in signaling reward magnitudes (Carlezon and Thomas, 2009). Pharmacological (Ikemoto and Wise, 2004) and optogenetic (Nieh et al., 2013) manipulation of NAc activity influences operant responding or consummatory behavior in rewarded tasks. Injection of μ-opioids into NAcS is especially effective at promoting feeding and affective indications of pleasure (Smith et al., 2011). NAcS lesions impede association of reward magnitudes with environmental cues (Albertin et al., 2000). Numerous electrophysiological studies have recorded activity dependent on expected reward value or reward receipt in NAc in rodents (Ambroggi et al., 2011; Nicola et al., 2004; Roitman et al., 2005; Setlow et al., 2003; Taha and Fields, 2005). Human fMRI studies have repeatedly implicated ventral striatum as one of a number of areas involved in processing rewards (Doya, 2008). A particularly close association of striatal activity with reward delivery events is suggested by evidence that this area signals immediate but not delayed ingested (McClure et al., 2007) or monetary (Hariri et al., 2006; McClure et al., 2004) rewards. Functional imaging in both humans and animals has also revealed strong NAc responses to injection of drugs of abuse (Knutson and Gibbs, 2007). Our results go beyond the earlier findings by singling NAc out, from an unbiased whole-brain study, as a locus where activity evoked directly by reward delivery correlates with a behavioral measure of subjective reward magnitude. This suggests in turn that neural population activity in NAc, more so than activity in distal input or projection fields, is particularly closely associated with hedonic aspects of reward.

HIGHLIGHTS.

• Multimodal experiments delineate brain regions engaged by rewarding stimulation

• Nucleus accumbens fMRI responses track psychometric reward magnitudes

• VTA responses and dopamine signals are dissociated from reward magnitudes

• Pharmacological inactivation of NAc disrupts reward titration curves

Abbreviations

Amyg

Amygdala

AmygAM

Anteromedial Amygdala

AmygPL

Posterolateral Amygdala

BOLD

Blood Oxygenenation-Level Dependent

CPu

Caudate-Putamen

CgCx

Cingulate Cortex

DpMe

Deep Mesencephalic Nuclei

DTI

Diffusion Tensor Imaging

fMRI

Functional Magnetic Resonance Imaging

EPI

Echo Planar Imaging

GLM

General Linear Model

GP

Globus Pallidus

Hippo

Hippocampus

LH

Lateral Hypothalamus

LS

Lateral Septal Nucleus

MFB

Medial Forebrain Bundle

NAc

Nucleus Accumbens

NAcC

Nucleus Accumbens Core

NAcS

Nucleus Accumbens Shell

Tu

Olfactory Tubercle

OCx

Orbital Cortex

PiCx

Piriform Cortex

PO

Preoptic Area

ROI

Region of Interest

SCx

Somatosensory Cortex

SI

Substantia Innominata

SN

Substantia Nigra

Thal

Thalamus

VP

Ventral Pallidum

vSub

Ventral Subiculum

VTA

Ventral Tegmental Area