Noradrenaline and Dopamine Neurons in the Reward/Effort Trade-Off: A Direct Electrophysiological Comparison in Behaving Monkeys

Abstract

Motivation determines multiple aspects of behavior, including action selection and energization of behavior. Several components of the underlying neural systems have been examined closely, but the specific role of the different neuromodulatory systems in motivation remains unclear. Here, we compare directly the activity of dopaminergic neurons from the substantia nigra pars compacta and noradrenergic neurons from the locus coeruleus in monkeys performing a task manipulating the reward/effort trade-off. Consistent with previous reports, dopaminergic neurons encoded the expected reward, but we found that they also anticipated the upcoming effort cost in connection with its negative influence on action selection. Conversely, the firing of noradrenergic neurons increased with both pupil dilation and effort production in relation to the energization of behavior. Therefore, this work underlines the contribution of dopamine to effort-based decision making and uncovers a specific role of noradrenaline in energizing behavior to face challenges.

Introduction

Neuromodulatory systems are fundamental for motivation, which shapes behavior as a function of expected costs and benefits. But what is the specific contribution of each neuromodulator? Even if noradrenaline (NA) is critical for motivation and decision making, its exact contribution remains unclear (Aston-Jones et al., 1991; Aston-Jones and Cohen, 2005; Yu and Dayan, 2005; Doya, 2008; Ventura et al., 2008; Bouret and Richmond, 2009). We recently proposed that NA was involved specifically in facing challenges, including effort (Bouret et al., 2012; Sara and Bouret, 2012; Bouret and Richmond, 2015). The role of NA could be complementary to that of dopamine (DA), which is traditionally thought to mediate the incentive effects of rewards on behavior (Schultz et al., 1997; Morris et al., 2006; Berridge, 2007; Phillips et al., 2007; Bromberg-Martin et al., 2010). However, the implication of DA in effort processing is debated (Gan et al., 2010; Salamone and Correa, 2012; Pasquereau and Turner, 2013; Hosking et al., 2015). We suggest that this controversy is due to the dual effect of effort on behavior: effort represents both a cost and a difficulty. It is a cost because information about upcoming effort decreases the value of the expected reward. When given the choice, we usually select the least effortful option for a given reward (Croxson et al., 2009; Day et al., 2010; Prévost et al., 2010; Massaro et al., 2012; Hosokawa et al., 2013), but it is also a difficulty in that, once we undertake an effortful action, we must mobilize the energy needed to face the physical challenge (Acevedo et al., 2007; Schmidt et al., 2009; Kurniawan et al., 2013; Meyniel et al., 2013).

We hypothesized that the dopaminergic system is primarily involved in processing effort as a cost by encoding information about upcoming effort and computing outcome value, which is used for the decision to engage in the action or not. Conversely, we also hypothesized that the processing of effort as a difficulty involves the noradrenergic system. In the framework proposed by Sara and Bouret (2012), upcoming challenges induce a coactivation of the autonomic system and of the locus coeruleus (LC), the main noradrenergic nucleus, enabling the organism to overcome the difficulty. In that framework, the NA system should be activated when animals engage into a difficult action and mobilize both the physical (muscles) and physiological (autonomic) energy required to face the effort challenge.

To test this hypothesis, we trained monkeys to perform a reward/effort task while we recorded the activity of single DA- and NA-producing neurons located in the substantia nigra pars compacta (SNc) and the LC, respectively. Our data indicate that dopaminergic neurons compute outcome value by combining reward and effort-cost information, whereas the activity of noradrenergic neurons increase when monkeys mobilize resources to meet a physical challenge and overcome a difficulty. Therefore, this work demonstrates a clear double dissociation between two aspects of effort, cost and difficulty, and the activity of two major neuromodulatory systems, DA and NA.

Materials and Methods

Subjects.

Three male rhesus monkeys (Monkey D, 11 kg, 5 years old; Monkey E, 7.5 kg, 4 years old; Monkey A, 10 kg, 4 years old) were used as subjects for the experiments. All experimental procedures were designed in association with the Institut du Cerveau et de la Moelle Épinière (ICM) veterinarians, approved by the Regional Ethical Committee for Animal Experiment (CREEA IDF no. 3) and performed in compliance with the European Community Council Directives (86/609/EEC).

Task and behavior.

The paradigm involved a physical effort task in which the subjects must squeeze a bar to reach a minimum of imposed physical force. The experimental design bears on two main factors that are orthogonalized: (1) the physical effort that the subject has to provide and (2) the reward size. There were nine experimental conditions (see Fig. 1b) defined by the three levels of effort required and three sizes of reward delivered. Within a session, the nine different experimental conditions were selected with equal probability and presented in a random order. An individual visual cue presented at the beginning of each trial indicates which of the nine conditions was selected. To perform a trial correctly, the monkey was required to fixate a red spot for at least 600 ms (750 ± 150 ms) until the cue appeared. After a variable delay (1500 ± 500 ms from cue onset), the red spot turned green (go signal,) indicating that the monkey had to initiate the response within 1000 ms to complete the trial. The response consisted in squeezing the bar with the minimum force level indicated by the cue at the beginning of the trial. When the monkey reached the expected force level, the point turned blue (feedback) and remained blue if the animal maintained the force above the threshold for 600 ± 100 ms to obtain the reward. If the monkey broke fixation before reward delivery, if it failed to exert the required force, or if it released the bar too early, all stimuli disappeared and an error was scored. All erroneous trials were repeated. On correctly performed trials, a liquid reward was delivered (one, two, or four drops of water). The intertrial interval was 1500 ± 200 ms.

Figure 1.

Experimental procedures. a, The task consists of squeezing a grip to obtain a reward. A trial started with a red point and the cue appeared within 900 ms after the animal initiated fixation. The trial was aborted immediately if the monkey broke fixation before reward delivery. After a variable delay of 1500 ± 500 ms, the dot turned green (Go signal), indicating that the monkey had to squeeze the bar strongly enough to reach the appropriate effort level, indicated by a blue point (feedback). If the monkey exerted enough force to score a correct response, it obtained the number of reward drops predicted by the cue. b, Experimental design and cue set used for the experiments: cues and background were isoluminant. Each trial corresponds to one of the nine conditions, defined by three levels of effort and three sizes of reward. c, Recording sites shown on MR images illustrating the position of electrodes targeting the SNc (orange dot) in Monkey E (left) and the LC (blue dot) in Monkey D (right). The middle image shows an example of simultaneous placement of recording electrodes in the LC (blue dot) and the SNc (orange dot). d, We used a pharmacological test to confirm our classification of SNc neurons as dopaminergic based on neurophysiological criteria. The firing of this SNc neuron fluctuates between 2 and 3 spikes/s before the intramuscular injection of apomorphine (0.05 mg/kg, i.m.). The injection induces a transient activation, presumably because of the arousing effect of the mechanical stimulation, and a lasting inhibition of about 20 min, presumably because of the stimulation of inhibitory autoreceptors on this neuron.

Pupil data and eye position were recorded using an eye-tracking system (ISCAN) that provided both analog voltages and direct digital information. All along the experimental procedures, focal distance and magnification were kept constant after calibration. Stimuli were controlled by two networked computers running the task (real-time experimentation and control or REX) and the visual stimuli (Presentation; Neurobehavioral Systems). Given the involvement of the pupillary response in our paradigm, the psychophysical aspects of the visual stimuli have been controlled carefully to minimize the variability in the amount of light between visual stimuli. The cues were created isoluminantly and light intensity across cues was further validated using a luminometer.

Electrophysiological recordings.

Upon MRI scanning, a surgical procedure was performed under general isoflurane anesthesia to place the recording chamber (2 cm diameter, CILUX plastic; Crist Instruments) and the head fixation post (1.5 cm diameter, CILUX plastic; Crist Instruments). The recording chamber was centered stereotaxically over the body of the LC and/or SNc according to the coordinates based on the presurgery MRI and oriented with an angle of 12.4° (for Monkeys D and A) and 25° (for Monkey E) in the coronal plane. At the start of each recording session, an oil hydraulic microdrive micromanipulator (Narishige) was mounted to the recording chamber. A 23 gauge sharpened guide tube (Crist Instruments) housing a tungsten steel electrode (Frederick Haer; impedance range, 0.4–4 MΩ) was used to puncture the dura. The electrode inside the guide tube was positioned using a stereotaxic grid (Crist Instruments) with holes 1 mm apart.

The electrophysiological signals were acquired, amplified (10,000×), digitized, and band-pass filtered (100 Hz to 2 kHz) using the OmniPlex system (Plexon). Neuronal spikes were visualized and classified using the online spike-sorting algorithm. Both filtered and unfiltered signals were saved for subsequent offline validation. Several criteria were used to ensure quality of recorded units: only waveforms for which the peak voltage was at least 3× greater than the noise level were included. Only recordings for which the minimal interspike interval was greater than the normal refractory period (3 ms) were included. Only well isolated and clearly identified single LC and SNc units were used for the analysis.

The LC and the SNc were located based on a combination of anatomical (see Fig. 1c) and physiological criteria. The LC was located using the following classical criteria: low rate of spontaneous activity (<4 Hz), broad waveforms (>2.5 ms), and a characteristic burst-pause response to brief auditory or tactile stimuli (e.g., clapping hands; Grant et al., 1988; Aston-Jones et al., 1994; Bouret and Richmond, 2009). In addition, if during the session the monkey became drowsy, the firing rate of the LC neurons decreased sharply. All of the recorded neurons were found in a region covered by only 1 grid hole and at depths that varied by no more than 1 mm. The SNc was localized using MRI and functional properties of adjacent neurons. Putative dopaminergic neurons were identified using previously established electrophysiological criteria: low firing rate during spontaneous activity (<8 Hz), broad waveforms (> 2 ms), a characteristic positive component at the end of the waveform, and a phasic response to unexpected stimuli. Neurons found in the SNc with high discharge frequencies (>20 Hz) were considered nondopaminergic and not recorded. Apomorphine tests were performed to verify that selected SNc neurons were dopaminergic. Apomorphine has been shown to suppress the spontaneous activity of dopaminergic neurons reversibly (Aebischer and Schultz, 1984). Neuronal activity was monitored for a few minutes and apomorphine (0.05 mg/kg, i.m.) was injected. All neurons tentatively identified as SNc neurons based on the previously defined criteria displayed a prolonged decrease in activity after apomorphine injection lasting tens of minutes (see Fig. 1d).

Data analysis.

As in previous experiments, we used the decision to perform the trial (or not) to assess the relative value of the different trial types. The monkeys reacted to the different cues with different accuracy rates, a feature that has been used extensively as an effective measure with which to investigate motivation in tasks in which the attentional demand was equivalent across conditions (Minamimoto et al., 2009b; Bouret and Richmond, 2010). By dissecting the accuracy rate into different error types, we sought to highlight the experimental conditions in which the monkeys chose not to respond. In fact, even if the subjects did not have the choice among the different experimental conditions, they could still forgo the trial by refusing to perform the physical effort. The goal of the error analysis was precisely to single out cases in which the animals refused to do the trial. To this end, we focused on errors that could be regarded as trials in which the monkey did not want to even try to perform the trial. We selected specific error types to consider the error rates as the proportion of trials in which the monkey did not engage in the effortful action. First of all, monkeys could not engage in the operant response at all. These errors represent the situation in which the monkey has maintained the fixation for the required time and has seen the cue indicating both reward size and effort level, but did not respond by squeezing the bar. Second, we selected erroneous trials in which the monkey did not engage in the operant response because it broke the fixation (if the monkey broke the fixation, all of the visual stimuli disappeared). Third, given the structure of the task, monkeys could decide to forgo the trials before the cue even appeared. Indeed, after an error, they knew that the trial would be repeated and they could abort it by breaking fixation early enough in the trial to prevent the cue from appearing. They could also decide to forgo the trial before the cue based on their internal state (e.g., because of fatigue or satiety) given the average expected amount of reward or effort. The predictions for these three kinds of errors were that the monkeys would refuse to engage in the effortful action when the value associated with this action was too low. By using error rates as our dependent measure, we infer the effect of expected reward size and predicted effort level associated to each experimental condition. Therefore, all of the error types that we selected were used as a measure of the action value associated with each experimental condition.

All of the recorded neurons identified as DA neurons for the SNc and NA neurons for the LC were included in the analyses without any kind of selection based on their activity in the task. All of the analyses were conducted using MATLAB software (The MathWorks). Only correct and nonrepeated trials were included in the following neuronal analyses. To identify significant neuronal responses, we applied statistical tests (paired t tests) to several analysis windows. The analyses were conducted on single units in the following windows: (1) the early cue activity from 0 to 500 ms from cue onset (compared with a baseline activity from −400 to 0 ms to cue onset), (2) the preaction activity from −150 to 0 ms before action onset (compared with a baseline activity from −400 to −150 ms before the action onset), and (3) the action activity from 0 to 500 ms from action onset (compared with a baseline activity from −400 to 0 ms before action onset). The latency of the neural response was defined as the first bin in which the firing rate was significantly different from the firing rate in the baseline window. The latency of the neuronal response was calculated only if it was significantly different from the activity in the baseline window.

The neural encoding of task variables by single units was quantified using the fitted coefficients from a generalized linear model (GLM) in which neuronal single-trial firing rates were modeled as a constant factor plus a weighted linear combination of three variables: the effort level, the reward size, and the trial number. The three variables were z-scored to get normalized regression coefficients and to facilitate the comparison of the effects across neurons. The firing rates were raw data expressed in spikes per second, but all results were replicated using z-scored firing rates (by structure). The neural encoding for analyses was quantified using the same statistical model used for single-unit analyses. Again, the three variables (reward, effort, and trial number) were z-scored and the firing rates were raw data. Second-level analyses were conducted as follows: the regression coefficients by neuron were computed using the above-mentioned GLM procedure and their distributions compared against zero (paired t test).

To assess the relationship between behavior and neuronal activity, the influence of the task factors on two variables, neuronal activity and behavior, was compared using two different analytical approaches (across all trials and across sessions). First, we used a mediation model analysis to explain the variance observed in choices with the neuronal activity of single cells. We ran a GLM to explain the variance observed in neuronal firing rate (fr) with three variables: trial number, expected reward and anticipated effort (fr = β trial + β reward + β effort). We then used the residuals of this GLM (fr*; thus removing the effects of task factors from the neuronal activity) to explain the choice behavior on a trial-by-trial basis. For this, we ran a logistic regression to explain the variance observed in the choices with four variables: residuals of firing rate, trial number, expected reward, and anticipated effort (choices = β fr* + β trial + β reward+ β effort). Second, we computed the expected net value of each trial by subtracting the effort level to the expected reward size (e.g., for a cue announcing effort level 1 and reward size 2, the expected net value was 1). The regression coefficients were then computed by neuron using a GLM procedure to explain firing rates and choices (logistic regression) with two variables: the trial number and the above-mentioned net value.

Results

Behavioral performance

The behavioral task (Fig. 1a,b) was designed to study two ways in which physical effort affects behavior: (1) the cost associated with the advanced information about effort production, which is reflected in the value of the corresponding option and the choice of the animals to engage in the corresponding course of actions; and (2) the difficulty of producing a physical effort, which, at the time of the effort itself, results in the mobilization of energy (physical and physiological) necessary to overcome the difficulty and reach the goal.

To assess the extent to which effort, as a cost, decreases the outcome value, we compared the willingness of the monkeys to engage in the task across the nine conditions (defined by the combination of three reward sizes and three effort levels). Even if erroneous trials were repeated, monkeys could decide to forgo the trial (mostly by breaking visual fixation; see Materials and Methods) if its expected outcome value was too low. The reward value predicted by the cue was clearly discounted by the information about upcoming effort and the progression through the session (Fig. 2a–c). Monkeys' choices to perform the trial were positively affected by the expected reward (GLM; see Materials and Methods; reward factor: β = 0.35, p < 10⁻¹⁰) and negatively affected by the anticipated effort (effort factor: β = −0.4, p < 10⁻¹⁰). Moreover, the animals' choices were negatively affected by the trial number (trial number: β = −0.3, p < 10⁻¹⁰) and the constant term was significant (constant: β = 0.42, p < 10⁻¹⁰; Fig. 2a–c). Monkey D forgoes 32 ± 2.44% of the trials (mean ± SEM), Monkey E forgoes 39 ± 2.3% of the trials, Monkey A forgoes 37 ± 2.3% of the trials. On average, the error rates were equivalent between LC and SNc recording sessions (unpaired t test, p > 0.05): monkeys forgo 36 ± 2.3% of the trials for the SNc recording sessions and 34 ± 2.4% for the LC recording sessions. Finally, we compared the regression coefficients for reward, effort, and trial number directly, as well as the constant term in a GLM to account for choices, and, on average, these coefficients were indistinguishable between the LC and SNc sessions (unpaired t test, p > 0.05). We thus conclude that the behavioral effects are consistent across LC and SNc recording sessions. To estimate the timing of the decision to forgo the trial after the onset of the visual cue, we measured the latency of the fixation breaks. Fixation breaks occurred within 500 ms after cue onset and were indistinguishable across monkeys (ANOVA, no effect of monkey, p > 0.05).

Figure 2.

Behavior in the task. a–c, Influence of trial number, effort level, and reward size on the choice to perform the trial for each of the three monkeys (Monkeys D, E, and A, respectively). After cue onset, monkeys could decide to abort the trial or to continue and exert the effort at the Go signal (choice to go). The influence of the three factors on the choice to go of each monkey was assessed using a logistic regression with a GLM. Parameter estimates are represented as the mean ± SEM of the regression coefficients across all sessions. For all three monkeys, the choice to perform the trial was influenced positively by the size of the expected reward (blue bar) and negatively by both the expected effort (red bar) and the progression through the session (black bar). The constant term was significant (green bar) for the three monkeys. d, Effect of expected effort on force production. Average force profile (±SEM) aligned on the action onset, broken down into the three effort levels: easy (yellow), intermediate (orange),and difficult (red). Monkeys used the information provided by the cues to adjust their behavior: they produced the minimum amount of force necessary to complete the trial (GLM, effort factor: β = 0.1, p < 10⁻¹⁰). e, Influence of effort on pupil area around the behavioral response. Average pupil diameter (±SEM) aligned on the response according to the three difficulty levels. f, Influence of experimental factors on the magnitude of pupil dilation response at the cue assessed using a GLM approach. Parameter estimates are represented as the mean ± SEM of the regression coefficients across all sessions. At cue onset, the pupil response was affected by the expected reward size, not by the expected effort or the progression through the session (trial number).

Animals readily used the information provided by the visual cue to adjust their force to the physical difficulty of the task: all three monkeys produced the minimum level of physical force on a trial-by-trial basis (Fig. 2d). In addition, there was a progressive decrease in the amount of force exerted over the session (GLM, trial number: β = −0.1, p < 10⁻¹⁰), but no effect of the amount of reward at stake. We assessed the underlying mobilization of physiological resources (autonomic arousal) using pupillometry after having reduced to a minimum the differences in luminance across conditions (see Materials and Methods; Fig. 1b). All 3 monkeys displayed a strong pupil dilation response at the onset of the action, and the magnitude of this pupil response increased with the level of the imposed physical effort (GLM, effort factor: β = 7.7 × 10⁻¹, p < 10⁻¹⁰; Fig. 2e). Moreover, we found a positive trial-by-trial correlation between the exerted force and the pupil dilation (r = 0.4, p = 10⁻¹⁶), even in the null space of the imposed effort factor (r = 0.2, p = 10⁻¹⁶). Indeed, to distinguish a direct correlation between exerted force and pupil dilation from a simple coactivation by the imposed effort factor, we regressed both force and pupil data on the effort level and used the residuals to determine whether their correlation was still significant after we had regressed out the imposed effort level. Therefore, force production involves not only the corresponding muscles, but also the autonomic system, which is consistent with the idea that effort production mobilizes energetic resources. The pupil response at the cue was strongly related to the amount of expected reward, but there was no effect of the amount of expected effort (GLM, reward factor: β = 2.5 × 10⁻², p = 2.7 × 10⁻⁶; effort factor: β = −2.2 × 10⁻², p = 0.7; Fig. 2f).

In summary, monkeys used the information provided by the cue to update their representation of the expected costs and benefits and decided accordingly whether they engaged in the trial or not. If they chose to engage, they furnished the required amount of physiological (autonomic) and physical (muscular) energy to produce the required force and obtain the expected reward.

Single-neuron activity

We recorded from 93 noradrenergic units in LC from Monkey D (n = 63) and Monkey A (n = 30) and 90 dopaminergic units in SNc from Monkey E (n = 31) and Monkey D (n = 59). The recording sessions included, on average, 187 ± 4.8 completed trials (mean ± SEM) for LC recording sessions and 190 ± 3.2 completed trials for SNc recording sessions. There was no difference in the activity of neurons recorded from different monkeys, so the data were pooled. The average spontaneous firing rate of LC neurons (2.6 ± 0.17 spikes/s) was significantly lower than that of SNc neurons (3.7 ± 0.47 spikes/s; p < 0.05, unpaired t test).

As shown in Figure 3, both LC and SNc neurons showed a fast transient activation at cue onset and around the action. We used a sliding window procedure to identify neurons displaying a significant increase in firing rate after cue onset and to measure the corresponding response latency. We observed a significant activation in response to the cue onset in 78/93 noradrenergic neurons and 80/90 dopaminergic neurons (no difference in proportion of responding neurons, χ² test: p > 0.05). The latencies of these phasic responses were indistinguishable between the 2 populations (SNc: 178 ± 11.4 ms, LC: 160 ± 11.9 ms; unpaired t test, t₍₁₅₆₎ = 1.17, p = 0.2). Around the time of the action, SNc and LC neurons also displayed a strong activation. Before the action, 84/93 LC neurons and 68/90 SNc neurons displayed a significant activation, with average latencies of −89 ± 5.8 ms and −111 ± 6.23 ms, respectively. The proportion of responding neurons was higher for LC compared with SNc neurons (χ² = 4.93, p = 0.03) and the activation occurred earlier in SNc compared with LC neurons (unpaired t test, t₍₁₅₅₎ = −2.6, p = 0.01). During the action itself, 83/93 LC neurons and 75/90 SNc neurons displayed a significant activation. The proportion of responding neurons was indistinguishable between the two populations (χ² = 4.93, p = 0.03), but the latencies of the action-evoked activations were shorter for LC compared with SNc neurons (unpaired t test, t₍₁₅₆₎ = 3.73, p < 0.001). In short, the latencies of cue-evoked responses were essentially the same in the two populations (Fig. 3). Before the action, the activation occurred earlier in the SNc but it was more pronounced in the LC. Conversely, during the action itself, LC neurons displayed a faster response compared with the SNc neurons.

Figure 3.

Neuronal activity around events of interest. Neuronal activity around cue onset (a–d, vertical gray line) and action onset (e–h, vertical green line). a, Spike activity (raster and spike density function, orange line) of a SNc unit showing a strong activation at the cue. b, Sliding window analysis showing standardized firing rate of all SNc neurons recorded (z-scored by neuron, color-coded) around cue onset (t = 0). Each line represents the activity of a single neuron and neurons are sorted by increasing latency of the peak. c, Activity of a single LC neuron activated after cue onset (same representation as in a). d, Sliding window analysis of the activity of all LC neurons recorded, aligned around cue onset (same representation as in b). e, Activity of a representative SNc neuron around action onset. This cell shows a strong activation before the action and a more limited activation during the force production itself. f, Sliding window analysis of the population of SNc neurons around the action onset. The peak of the activation of SNc neurons can occur before or after the action onset. g, Activity of a single LC neuron around action onset. Note the clear double activation, with one peak before the action and one during the effort. h, Sliding window analysis of the population of LC neurons around action onset. Almost all LC neurons are activated both before and during the effort.

We then examined the influence of task parameters on the responses of these two populations. As shown for representative neurons in Figure 4, the magnitude of the phasic activations at the cue (Fig. 4a1,c1) and around the action (Fig. 4b1,d1) were significantly modulated across task conditions and these modulations differed markedly between LC and SNc neurons. To evaluate the influence of task parameters on firing rate, we compared spike counts across the different conditions using a GLM (see Materials and Methods) with three regression variables: reward size, difficulty level, and trial number.

Figure 4.

Modulation of neuronal activity by reward and effort factors. a, Activity of SNc neurons at the cue. a1, Activity of a representative SNc unit (same representation as in Fig. 3a) across the nine conditions defined by three levels of effort (increasing from top to bottom) and the three levels of reward (increasing from left to right). The response magnitude increases with the reward and decreases with the expected effort. a2, Distribution of regression coefficients for reward (blue) and effort (red) based on a GLM analysis of cue evoked firing rates (from 0 to 400 ms after cue onset) for each SNc neuron of the population (n = 90). The distribution of the effort regression coefficients is significantly shifted toward negative values as indicated by the red point (mean) and horizontal bar (SEM) above the distribution. The distribution of the reward regression coefficients was shifted toward positive values, as indicated by the blue point (mean) and horizontal bar (SEM) above the distribution. a3, Averages of regression coefficients (solid bars) and SEM (error bars) across trials for all SNc neurons (n = 90) at the time of the cue onset. b, Activity of SNc neurons at the action. b1, Activity of the same representative SNc unit (same representation as in a1). It is clearly activated in relation to the action, but the response is equivalent across all conditions. b2, Distribution of the regression coefficients for action-related firing rates (from −100 to 400 ms from action onset) for each neuron of the SNc population (same representation as in b). The distributions of the reward and effort coefficients are both centered on zero, indicating that on average reward and effort do not modulate SNc activity during the action. b3, Averages of regression coefficients across trials for SNc neurons (n = 90) at the time of the action onset (same representation as in a3). c, Activity of LC neurons at the cue. c1, Activity of a representative LC unit across the nine conditions (same representation as in a1). The activation of this neuron increases with the reward, but there is no effect of effort. c2, Distribution of the regression coefficients of cue evoked firing rates for each LC neuron of the population (same representation as in a2). The distribution of the reward coefficients (in blue) is shifted toward positive values, whereas the distribution of effort coefficients is centered on zero. c3, Averages of regression coefficients across trials for LC neurons (n = 93) at the time of the cue onset (same representation as in a3). D, Activity of LC neurons at the action. d1, Same LC unit as in c1. The activation of this neuron increases with the amount of effort, both before and after action onset. d2, Distribution of the regression coefficients of action-evoked firing rates for each LC neuron of the population (same representation as in a2). The distribution of the effort coefficients (in red) is significantly shifted toward positive values, but the distribution of reward coefficients is centered on zero. d3, Averages of regression coefficients across trials for LC neurons (n = 93) at the time of the action onset (same representation as in a3). *p < 0.05; **p < 0.01.

At the cue onset, second-level analyses reveal that the expected reward had a significant positive influence on the firing rate of both LC and SNc neurons (Fig. 4a2,c2; second-level analysis: mean beta distributions significantly greater than zero; paired t test: p < 0.01). Conversely, the distribution of effort regression coefficients was significantly smaller than zero for SNc but not for LC (Fig. 4a1,c2; paired t test: p = 0.005 and p = 0.153, respectively). In addition, we compared directly the distribution of effort regression coefficients between SNc and LC neurons and they were significantly different (unpaired t test, p < 0.05). Therefore, the anticipated effort cost announced by the visual cue had a stronger negative influence on SNc (Fig. 4a2) than on LC neurons (Fig. 4c2). Next, we tested the influence of advancement through the session (trial number), the other factor that had a negative influence on choices to perform the trial. This factor only had a significant effect on SNc activity (second-level analysis: paired t test: p = 0.04 and p = 0.4 for SNc and LC, respectively). However, we also compared directly the distribution of regression coefficients for trial number between SNc and LC neurons and they were not significantly different (unpaired t test, p = 0.5). Therefore, the progression through the session had a similar negative influence on SNc and LC neurons.

We also examined the influence of task parameters on individual neurons. At the cue onset, the influence of the expected reward on cue-evoked activity was significantly positive (p < 0.05) for 19/90 SNc neurons and 15/93 LC neurons and significantly negative (p < 0.05) for 9/90 SNc neurons and 2/93 LC neurons. The anticipated effort level had a positive influence on firing rates for 1/90 SNc neurons and 5/93 LC neurons and a negative influence on 9/90 SNc and 6/93 LC units. Trial number had a negative effect on the firing of 29/90 SNc and 30/93 LC neurons and a positive effect on 18/90 SNc and 20/93 LC neurons.

Around the action onset, the positive modulation by the effort difficulty level was much more pronounced in the LC than in the SNc. Before the action onset, the distribution of effort regression coefficients was significantly shifted toward positive values for the LC, but not for the SNc (second-level analysis; paired t test: p = 0.02 and p = 0.7, respectively). During the action, the distribution of effort regression coefficients was significantly shifted toward positive values for the LC (Fig. 4d2), but not for the SNc (Fig. 4b2). A second-level analysis confirmed that mean beta distributions were significantly greater than zero for LC but not for SNc (paired t test: p = 0.001 and p = 0.9, respectively). A direct comparison between regression coefficients for effort between SNc and LC neurones showed that, on average, they were significantly different (unpaired t test, p < 0.05). We also examined the influence of physical difficulty on the activity of single units. Right before the action onset, 26/93 LC and 17/90 SNc neurons increased their firing rate as the effort level increased (p < 0.05). During the action itself, 47/93 LC and 29/90 SNc individual neurons showed a significant positive modulation by the effort level (p < 0.05).

In short, right after the cue onset, the firing modulation of the two regions showed a common positive modulation by the size of the expected reward and a common negative modulation by the progression through the session, but the negative influence of expected effort on firing rates was significant only in the SNc and not in the LC. Conversely, around the action, the positive influence of effort on firing was stronger in the LC than in the SNc.

DA and value computation

Single-unit analyses showed that, at cue onset, SNc neurons positively encoded the discounted outcome value when monkeys used that information to decide whether they would perform the trial. The previous analysis describes the population as a collection of individual neurons and captures the effects that are coherent across all of the neurons. We computed a complementary population analysis by pooling all of the spikes of all of the neurons together to assess the global output of these structures (see Materials and Methods). The expected reward modulates positively the cue-evoked activity of the resulting population firing of SNc neurons (GLM, reward factor: β = 0.06, p = 1.3 × 10⁻¹²; Fig. 4a3). The two factors that had a negative influence on value and decision, effort cost and trial number, had a significant negative effect on the population activity of SNc neurons (GLM, effort factor: β = −0.03, p = 0.003; trial number: β = −0.04, p = 1.75 × 10⁻⁶; Fig. 4a3). The dynamics of these effects are shown in Figure 5a.

Figure 5.

Population activity. a, Top, Population average activity (± SEM, shaded areas) of SNc neurons around the cue onset, broken down into the three reward conditions. The magnitude of the cue-evoked activation increases with the expected reward. a, Bottom, Population average (± SEM) of SNc neurons around cue onset, broken down into the three effort conditions. The magnitude of the activation decreases with the anticipated effort. b, Top, Population average SNc neurons around action onset sorted across the three reward conditions. The activity is not influenced by the expected reward size. b, Bottom, Population average of SNc neurons around action onset sorted across the three effort conditions. The firing increases with the expected effort, but only during the action itself. c, Top, Population average of all recorded LC neurons around cue onset sorted across the three reward conditions. The magnitude of the activation is greater in the high reward condition. c, Bottom, Population average of LC neurons around cue onset sorted across the three effort conditions. The activity shows little effect of effort. d, Top, Population average of LC neurons around action onset sorted across the three reward conditions. There is little difference across reward conditions. d, Bottom, Population average of LC neurons around action onset sorted across the three effort conditions. Action-evoked activity shows a strong positive modulation by the effort level both before and during the action.

Consistent with the analysis of individual neurons, population activity of SNc neurons did not display any effect of the task parameters right before the action onset (p > 0.05). On the contrary, and in contrast to the second-level analysis, in which the trend was not significant, we found a positive effect of the difficulty level after the action onset (Fig. 4b3) once the monkeys had already initiated the force production (GLM, effort factor: β = 2.3 × 10⁻², p = 2.69 × 10⁻⁶). The dynamics of these effects are shown in Figure 5b.

DA value-related signal reflects the value-driven behavior

The results illustrated thus far suggest that the activity of SNc neurons is directly related to the computation of the effort–reward trade-off used for the choice decision. To address this issue directly, we conducted a series of analyses to examine more closely the relationship between the firing rates of SNc neurons and choices. First, we ran a basic correlation analysis between the firing rates of SNc neurons and the subjects' choices: we included all of the trials of all of the SNc neurons and correlated the evoked spike count with the choice to perform the trial or not. That correlation did not reach significance (r = 0.06, p = 0.24). Second, we used a mediation model to explain the variance observed in choices with the neuronal activity of SNc cells: after removing the effects of expected reward and effort from the SNc activity (see Materials and Methods), we used the residuals of SNc firing rate to explain the choices. The mediation analysis led to an insignificant result (p > 0.05), meaning that SNc firing rate did not mediate directly on a trial-by-trial basis the effects of this task on choice behavior.

Next, we examined the relationship between intersession variability in choice behavior and the firing of SNc neurons in relation to the task parameters (effort level, reward size, and trial number). To this end, we computed an expected net value for each trial (the reward size discounted linearly by the effort cost; see Materials and Methods) and used this value index to explain the observed choices and the neuronal firing rates for each recording session. We then compared the effect of the expected net value on choice behavior versus the effect of expected net value on SNc cue-evoked firing across sessions. As shown in Figure 6a, there was a significant positive correlation between the weights of these two regressions (r = 0.25, p < 0.01) across recording sessions. This positive correlation between regression coefficients indicates that the reward/effort trade-off affects the subjects' decision to perform the trial and SNc neurons alike. We ran the same analysis for LC neurons, but did not observe any significant relationship between LC activity and choice behavior across sessions (r = 0.03, p = 0.7).

Figure 6.

Neuronal activity and behavior. a, The influence of net value on choices is correlated with its influence on SNc cue activity (each dot represents an SNc neuron). Correlation between the regression coefficients of net value (reward size-effort level) on cue evoked activity of SNc neurons (trial-by-trial, ordinates) and the regression coefficients of net value on the choice to perform the trial (trial-by-trial, abscissa). There is a significant positive correlation between these two regression coefficients across SNc neurons (n = 90), indicating that the more SNc neurons are activated by the net value, the more likely the monkey is to engage in the trial. b, Trial-by-trial relationship between the firing of LC neurons at the action and the amount of exerted force (z-scored by session). Data were binned for display (800 trials per bin ± SEM, error bars). The activation of LC neurons strongly correlated with the amount of force produced during the action. c, Trial-by-trial relationship between the firing of LC neurons at the action and the magnitude of the pupil dilation response (z-scored by session). Data were binned for display (800 trials per bin ± SEM, error bars). The activation of LC neurons was strongly related to the action-related pupil dilation, which provides a proxy for the underlying autonomic activation. In all plots, lines represent robust regression fits.

To investigate the dopaminergic response dynamics, we compared the timing of SNc responses with that of the average decision to forgo the trial (∼450 ms after cue onset; see above). The positive reward signal reached its peak, on average, 210 ms after the cue onset and the negative effort signal peaked, on average, 220 ms after the cue onset, suggesting that the value is encoded by SNc neurons before the decision to perform the trial, which occurs >400 ms after cue onset.

NA, difficulty, and resources engagement

The positive relationship between LC activity and difficulty level during the action was confirmed at the population level, (GLM β = 9.7 × 10⁻², p < 10⁻¹⁶; Fig. 4d3). Moreover, there was a significant relationship among LC population activity, the amount of force produced (Fig. 6b; r = 0.13, p < 0.01), and the associated pupil dilation, which reflects the autonomic activation (Fig. 6c; r = 0.08, p < 0.01). We complemented the analysis by looking at the correlation coefficients across all recorded LC neurons: the distributions of the correlation coefficients were significantly greater than zero both for LC-pupil and LC-exerted force correlations (paired t test, p < 0.01). Importantly, these correlations held even after we removed the influence of the effort factor by looking at the correlation between the residuals of the imposed effort level (r = 0.03, p < 0.01 and r = 0.009, p < 0.01, respectively). Therefore, the relationship between LC activity and the mobilization of physical and physiological energy is not a mere artifact of the task. The dynamics of these effects are shown in Figure 5d. The relationship between LC activity and effort appears before the actual movement (GLM β = 0.03, p = 0.001). In addition, during that phase preceding the action, LC activity displayed a small but significant negative effect of the expected reward (GLM,: β = −0.02, p = 0.04). This relationship among force, pupil dilation, and firing rates was specific to LC neurons: the positive correlation between action-related SNc activity and the amount of force produced (r = 0.04, p = 0.04) disappeared after regressing out the effort factor. In other words, unlike in the LC, this small correlation in the SNc is mostly driven by the task. Furthermore, there is no correlation between the activity of SNc neurons and the intensity of the pupil response at the population level (r = 0.002, p > 0.05) and the distribution of the correlation coefficients across all recorded SNc neurons was not significantly different from zero (paired t test, p = 0.8).

The strong relationship between LC activity and pupil dilation was not restricted to the action. Consistent with single neuron analysis, the population activity of LC neurons at cue onset scaled positively with the size of the expected reward (GLM β reward = 0.03, p < 10⁻¹⁰; Figs. 4c3, 5c, top), just like the pupil response (GLM β reward = 0.05, p < 10⁻⁴; Fig. 2f). Moreover, the LC population activity at the cue onset decreased throughout the session (β trial number = −0.01, p < 0.05), whereas we did not find any effect of the expected effort level (β effort = 0.003, p = 0.7; Figs. 4c3, 5c, top). The correlation between LC activity and pupil dilation at the cue was significant (r = 0.17, p < 0.01). Importantly, this relationship between LC activation and pupil dilation was not a mere side effect of the reward information because it was still significant after having removed the influence of reward on each of these variables (i.e., it was also significant between the residuals of the reward factor; r = 0.02, p < 0.05). This effect was specific to LC neurons because the firing of SNc neurons at the cue did not correlate with the magnitude of the pupil response (p > 0.05).

Discussion

This work captures two intuitions about effort: it is a cost, in that it decreases the value of the associated outcome, and it is a difficulty, in that it makes the action more challenging to perform. Monkeys decided to forgo trials requiring high effort costs and small rewards. Our neuronal data indicate a specific implication of SNc neurons in coding such value-based decision making. Conversely, when monkeys committed to perform the trial, they faced the imposed difficulty by producing the required amount of force in conjunction with an activation of both LC neurons and the autonomic system. This suggests a specific implication of NA neurons in mobilizing resources to energize behavior and face the challenge at hand.

SNc and decision value

Even if both SNc and LC neurons were activated shortly after cue onset, which provided information about costs (effort level) and benefits (reward size), they showed a different sensitivity to this information. The firing of SNc neurons increased with the size of the expected reward and decreased with both the effort cost and the progression through the session, the two factors that had a negative effect on decision value. Even though LC neurons showed a similar response pattern, the negative effect of effort was significant only for SNc neurons and the difference in sensitivity to effort between the two structures was significant. Moreover, the effect of expected net value on SNc firing correlated with the effect of expected net value on choices, which is consistent with the idea that SNc firing relates to the subjective outcome value that governs the decision to perform the trial or not (Morris et al., 2006; Croxson et al., 2009; Day et al., 2010; Lak et al., 2014). The activity of SNc neurons, however, preceded the overt choice and was not related directly to the motor aspects of the task, suggesting a closer association with evaluation processes upstream from the decision. This could account for the apparent discrepancy between our work and a recent study by Pasquereau and Turner (2013). Indeed, those investigators found little influence of effort on DA activity, but they used a task in which the physical effort affected mostly the difficulty of the motor performance rather than the decision to forgo the trial (Pasquereau and Turner, 2013). Last, the comparison of cue activity between the two structures emphasizes a clear dissociation between SNc and LC activity, the latter being more closely associated with the arousal level and the corresponding salience of the stimulus (Foote et al., 1980; Abercrombie and Jacobs, 1987; Sara and Segal, 1991; Bouret and Sara, 2004).

LC and facing challenges

The LC/NA system was specifically involved in another aspect of effort: the mobilization of energy necessary for the action. Indeed, even if both SNc and LC neurons were activated around the time of action, consistent with several recent reports (Bouret and Sara, 2004; Clayton et al., 2004; Bouret et al., 2012; Kalwani et al., 2014), the relation between neuronal activity and effort was much more pronounced in the LC. Importantly, the firing rate of LC neurons increased, not only with the amount of physical force produced, but also with the associated pupil dilation. Moreover, the close relationship among LC activity, pupil diameter, and force produced was still significant after removing the impact of the task parameters, which implies that this phenomenon exists over and above the task requirements. The activation of the autonomic system during effort production is a well known phenomenon, reflecting the physiological processes involved in providing the effectors (muscles) with metabolic energy (glucose and oxygen) (Collet et al., 1996; Acevedo et al., 2007). A similar autonomic activation has also been reported for attentional effort (Kahneman and Beatty, 1966; Howells et al., 2010; Webb et al., 2010). In that framework, one interpretation of LC activation is that it is related to the energetic need (in terms of both autonomic activation and muscular contraction) required to overcome the difficulty of completing the trial. Note that this effort-related activation of LC neurons could influence the early stages of the movement, but not the decision to initiate the action, because the modulation of LC activity started 100 ms before the action onset.

This interpretation of LC activity in terms of difficulty, with the idea that it is intimately related to the mobilization of energy to cope with a challenge, could also account for the response to the cue. Indeed, the classical interpretation of the orienting response involves mobilizing resources to deal with the unexpected stimulus (Kupalov, 1961; Aston-Jones et al., 1991, 1996; Steiner and Barry, 2011; Sara and Bouret, 2012). This is consistent with the general idea that LC activation facilitates sensory processing and mobilizes attentional resources (Waterhouse et al., 1998; Arnsten, 2000; Berridge and Waterhouse, 2003). In other words, the early activation of LC at the cue would facilitate the mobilization of sensory and attentional resources necessary to deal with the processing of the upcoming information.

Two systems for motivation: a forward incentive system and a reactive difficulty system

We have shown here that, even though they share strong similarities, the activity of SNc and LC neurons in this task might have complementary roles in motivation. The DA system clearly plays a role in value-based behavior. The expected outcome value modulates the firing of SNc neurons before it is manifested behaviorally (in terms of choice). Moreover, the modulation of SNc firing decreases significantly after the decision to engage in the action has been taken. This underlines the implication of the DA system in modulating the future behavior as a function of advance information on outcome value (Morris et al., 2006; Berridge, 2007; Croxson et al., 2009; Lak et al., 2014). Indeed, DA influence is particularly strong in frontostriatal circuits involved in action planning and execution (Haber, 2003; Haber and Knutson, 2010).

The role of the NA system in motivation would be complementary in that it is reacting to information about an existing challenge rather than to information about future outcome. Indeed, the modulation of LC activity reflected the difficulty of the task at hand: processing the cue information at its onset and producing the effortful action once the decision had been taken. The release of NA might facilitate the mobilization of resources necessary to overcome the challenge, either by enhancing sensory-motor processes at hand and/or by facilitating the reset of functional networks in the forebrain (Berridge and Waterhouse, 2003; Aston-Jones and Cohen, 2005; Bouret and Sara, 2005; Yu and Dayan, 2005; Sara and Bouret, 2012). This contribution of LC to mobilizing resources to face a challenge is reminiscent of activity in two other regions: the centromedial thalamus and the anterior cingulate cortex. Just like the LC, centromedial thalamic neurons are very closely associated with arousal and orienting responses (Glenn and Steriade, 1982; Minamimoto and Kimura, 2002). In addition, they show a stronger activation around the decision in unrewarded trials, just like LC neurons in reward schedule tasks (Minamimoto et al., 2005; Bouret and Richmond, 2009; Minamimoto et al., 2009a; Bouret et al., 2012). A similar intuition could also capture the role of the cingulate cortex, which is critically involved in adjusting behavior as a function of effort (Walton et al., 2002, 2003, 2009; Heilbronner and Platt, 2013; Hosokawa et al., 2013; Parvizi et al., 2013). Therefore, the LC/NA system might belong to a larger system involved in mobilizing resources in situations of emergency to face the challenge at hand.

In conclusion, our work indicates that the two catecholaminergic neuromodulatory systems have a distinct but complementary role in effort-related motivation: the activity of SNc neurons tracks the value of future outcomes and DA release in target regions orients the behavior toward the least effortful options. The activity of LC neurons goes along with the autonomic response to current challenges and NA release in target regions would enable the organism to face the difficulty at hand.