Anticipatory Energization Revealed by Pupil and Brain Activity Guides Human Effort-Based Decision Making

Abstract

An organism's fitness is determined by how it chooses to adapt to effort in response to challenges. Exertion of effort correlates with activity in dorsomedial prefrontal cortex (dmPFC) and noradrenergic pupil dilation, but little is known about the role of these neurophysiological processes for decisions about future efforts, they may provide anticipatory energization to help us accept the challenge or a cost representation that is weighted against the expected rewards. Here, we provide evidence for the former, by measuring pupil and functional magnetic resonance imaging (fMRI) brain responses while 52 human participants (29 females) chose whether to exert efforts to obtain rewards. Both pupil-dilation rate and dmPFC fMRI activity increased with anticipated effort level, and these increases differ depending on the choice outcome: they were stronger when participants chose to accept the challenge compared with when the challenge was declined. Crucially, the choice-dependent modulation of pupil and brain-activity effort representations were stronger in participants whose behavioral choices were more sensitive to effort. Our results identify a process involving the peripheral and central human nervous system that simulates the required energization before overt response, suggesting a role in guiding effort-based decisions.

SIGNIFICANCE STATEMENT The brain's arousal system tracks the effort we engage in during strenuous activity. But much less is known about what role this effort signaling may play when we decide whether to exert effort in the future. Here, we characterize pupil-linked arousal and brain signals that guide decisions whether to engage in effort to gain money. During such choices, increases in brain activity and pupil dilation correlated with the effort involved in the chosen option, and these increases were stronger when people decided to accept the effort compared with when they rejected it. These results suggest that the brain arousal system guides decisions by energizing the organism for the prospective challenge.

Introduction

Should I go to the gym or skip training? Trade-offs between effort and reward are commonplace. The ability to choose between high-cost, high-yield and low-cost, low-yield actions is crucial for survival in all animals (Bautista et al., 2001). Dopaminergic (DA) reward signals have long been identified to play a pivotal role in guiding choices (Schultz et al., 1997; Schultz, 2002; Niv et al., 2005; Ostlund et al., 2011; Bartra et al., 2013; Beierholm et al., 2013; Varazzani et al., 2015; Walton and Bouret, 2019). By contrast, it is less clear how effort signals guide decisions. Neural effort signals in the locus coeruleus noradrenergic (LC-NA) arousal system (Zénon et al., 2014; Varazzani et al., 2015) and frontoinsular network (Prévost et al., 2010; Kurniawan et al., 2013; Meyniel et al., 2013; Skvortsova et al., 2014; Hauser et al., 2017; Arulpragasam et al., 2018; Aridan et al., 2019) scale monotonically with increasing task difficulty (McGuire and Botvinick, 2010), but how these processes functionally contribute to choice is less understood.

Two possible roles of effort signals have been proposed. First, a prevailing view in decision theory posits that efforts incur costs that are by definition subtractive; that is, increasing costs lower the motivational value of given rewards (which is calculated as perceived reward minus perceived cost; Hull, 1943). Consistent with this view, functional magnetic resonance imaging (fMRI) studies show net value signals for reward that are subjectively “discounted” by effort (Prévost et al., 2010; Burke et al., 2013; Klein-Flügge et al., 2016; Chong et al., 2017; Arulpragasam et al., 2018; Aridan et al., 2019; Bernacer et al., 2019). However, these signals reflect a combination of both the rewarding and effortful aspects of the choice options, which impairs direct interpretations of how much these signals are truly effort-driven and how effort per se may affect the choice process.

Second, consistent with the idea that effort requires resource mobilization (Hockey, 1997), effort signals may represent the energization needed for the action under consideration to be successfully achieved (Paravlic et al., 2018). An “energization” signal is defined to have an additive effect on the decision value, since it is thought to enhance the readiness to take on a given effort (i.e., reduce the perceived effort cost) and thereby increase the reward–cost decision value. A sizeable literature indicates that LC-NA activity plays an important role in changing arousal states (Takahashi et al., 2010; Pfaff et al., 2012; Poe et al., 2020) by providing neuromodulatory input to the entire neocortex (Porrino and Goldman-Rakic, 1982; Chandler et al., 2013; Schwarz et al., 2015), thereby facilitating energization (Varazzani et al., 2015; Jahn et al., 2018). NA activity can directly influence pupil size and is linked to pupil dilation (Joshi et al., 2016; Reimer et al., 2016; Gelbard-Sagiv et al., 2018), making phasic, task-related pupil an accurate indicator of brain arousal states (McGinley et al., 2015; Yüzgeç et al., 2018). However, it remains unclear whether the effort signals that guide choices would draw on the same pupil-linked arousal system that facilitates actual behavior energization (Zénon et al., 2014; Varazzani et al., 2015; Xiang et al., 2019; Borderies et al., 2020).

Teasing apart these two scenarios is not trivial. One effective way forward is to investigate how signals that scale with effort differ depending on choice outcome (Kurniawan et al., 2010), namely, for decisions to engage in effort versus decisions to forego effort. In a cost scenario, stronger brain signals for increasing effort (after controlling for rewards) by definition represent higher encoded costs, which decrease the net value and push individuals toward declining the offer. Consider an offer of 10 reward tokens for level 8 effort. A brain signal that represents “cost” information should be higher when this offer is rejected (the encoded cost of effort 8 dominates the reward value of 10 tokens) compared with when the same offer is accepted (the encoded cost of level 8 is dominated by the encoded reward value of 10 tokens). Thus, a cost scenario would predict a steeper neural effort signal in reject compared with accept decisions. Statistically, this would be expressed in a higher regression weight for effort when we choose to reject effort (“no” decision) compared with when we choose to engage in it (“yes” decision). In an energization scenario, by contrast, a higher effort signal would represent readiness to mobilize resources and therefore have an opposite impact on choice, tipping individuals toward a “yes” decision. The energization scenario would therefore predict the opposite pattern of steeper effort-related signals (i.e., higher regression weights) in “yes” compared with “no” decisions (Fig. 1A). In principle, decision values would also change as a function of how rewards (rather than efforts) are coded, but we control for this in our analyses by focusing on how the slopes of effort coding change after controlling for reward effects.

Figure 1.

Predictions, task design, and key measures. A, Three possible patterns of anticipatory neural responses to effort. Left, Signals coding for effort per se would scale monotonically with effort regardless of choice. Middle, Signals coding for the decision cost associated with effort should be steeper across effort levels when individuals reject the effort. Right, Signals coding the anticipatory energization needed to accept the challenge should be steeper across effort levels when individuals accept the effort. B, Experimental paradigm. Prescan: participants received visually-guided effort training on a hand-held dynamometer. Levels 1–9 correspond to 10–90% MVC. In the fMRI scanner, participants chose between an effortful option associated with variable amounts of reward and effort and a non-effortful option with smaller reward. Postscan: outside the scanner, eight trials were randomly selected and participants executed the effort they chose in those trials to obtain the reward. C, Behavioral effort sensitivity. This individual measure was derived by calculating for each participant the slope of the probability to choose the effortful option across effort levels. D, Phasic pupil measure. Grand-mean of pupil width during decision-making showed a stereotypical dilation shortly following stimulus onset, peaking right after averaged response onset (purple line), and constricting down to baseline level around stimulus offset. Pupil rate (z/s) was calculated by subtracting pupil width at response from pupil width at stimulus onset, divided by response time (RT). stim=stimulus; resp=response.

Here, we apply this experimental logic using an effort discounting task in an fMRI setting, while simultaneously tracking pupil dilation, a putative marker for LC-NA firing. This combination allows us to investigate systematically to what degree the brain arousal system may encode anticipated effort during decision-making as a cost or energization signal.

Materials and Methods

Participants

Fifty-two right-handed participants [29 females, mean age = 22.3 (3) years] volunteered to participate in this study. We determined the sample size using power analysis based on the small to medium effect size (d = 0.2–0.5) reported in past studies in the laboratory relating pupil size and biases in choice behavior. Participants received between 80 and 100 CHF (depending on the realized choices and performance) for their participation. Participants were screened for MRI compatibility, had no neurologic or psychiatric disorders, and needed no visual correction. Data from one subject were excluded because of eye-tracker data loss. Inclusion of this subject in the behavioral analysis did not change the statistical results, but for consistency, we excluded this data set from all analyses. We screened subjects based on their p_{(choose effortful)} to be within 0.1 and 0.9, and excluded one subject whose choice rate was 0.95. The final N was 49.

Procedure

Force calibration

Upon arrival, participants sat in the behavioral testing room, filled the MRI screening and consent forms, and received general instructions on the force task and MRI safety. Maximum voluntary contraction (MVC) level for each hand was obtained by averaging the top 33% force values produced during three 3-s squeezes. Continuous encouragement was given vocally during each entire squeeze period (e.g., “keep going, keep it up”).

Force training

Guided by a vertical bar on-screen (Fig. 1B), participants were trained to do hand squeeze sets at levels 10–90% MVC (displayed as levels 1–9). This dynamometer effort task mimics a typical hand force exercise at the gym, with a cycle of repetitions (“reps”) of muscle contractions (3 s) and relaxations (3 s) for each level. To prevent muscle fatigue, these were done alternating between left and right hand. During training, one set consisted of five repetitions and there were in total 10 squeeze sets (10 × 5 = 50 reps) to be evaluated by a certain criterion. Levels 1–8 were presented once, pseudorandomly assigned to either left and right, and level 9 twice, once for each hand. The order of force levels was also pseudorandomized. Half of the subjects practiced on levels 1, 3, 5, 7, 9 with left hand and 2, 4, 6, 8, 9 with right hand, and vice versa for the other half of subjects. The criterion was to maintain force above the target for at least two of the 3-s rep (non-consecutively). At the end of each training round, participants received a summary of their performance and were asked to repeat each unsuccessful force production. Overall, all participants underwent at most three training rounds (M = 2.22, SD = 0.46). After the last round, 38 participants successfully completed all 50 reps (0 unsuccessful rep), whereas 11 participants had a few unsuccessful reps (M = 4.3%, SD = 3.5%). These results suggest that the training was very successful.

Following a 5-min break, they proceeded with a subjective rating task in which they had to squeeze for each hand once at levels 1, 3, 5, and 9 for 5 s without knowing the difficulty levels. They were told that in some trials it would be easy to raise the bar to reach the target, which in this task was always displayed at the midline, while in other trials it would be harder to do it. After each 5-s squeeze, they then rated on a continuous visual analog scale how effortful the grip was for them. They were instructed that the leftmost and rightmost point in the scale should refer to level 0 (merely holding the dynamometer) and level 10, respectively. The force training was successful as indicated by a close relationship between subjective and objective effort, mean Pearson's r = 0.93, SEM = 0.0073, t₍₄₆₎ = 127.63, p < 0.0001.

Before scanning, participants made five practice decisions and we made sure that participants fully comprehended the task. The effort discounting task was done in the fMRI scanner. Participants were aware that the effort they were considering now consisted of one set of 10 reps (instead of five). To prevent participants from taking decisions based on anticipated muscle fatigue, only a random selection of eight decisions were actually realized in the behavioral testing room after the scan, and participants were fully aware of this. Participants filled some questionnaires, were debriefed, given payment, and thanked for their participation before leaving the lab.

Effort discounting task

In the scanner, participants were given a series of choices between an effortful and a non-effortful option. On each trial, the effortful option entailed varying effort (one of six levels, levels 4–9) and reward amounts (one of six levels, 0.5–10 CHF; Fig. 2A). The non-effortful option entailed minimal effort (fixed at level 1) and a lower reward amount (30% or 40% of the reward amount of the effortful option), giving a clear incentive to choose the non-effortful option if the larger effort was not worth the reward. To rule out risk as a potential confound (namely that accepting a level 9 offer gives a higher risk of failure compared with accepting a level 4) we ensured that the effort training at all levels was successful (overall failure rate during training was <1%, M = 0.9%, SD = 2.4%).

Figure 2.

Behavioral and pupil results. Choice proportions (A) and RT (C) as a function of reward and effort associated with the effortful option. B, Weights of logistic regression of choice on reward, effort, and the interaction from a standard model based on the offers the participants see on the screen. D, Weights of multiple regression of RT on reward, effort, and the interaction from a standard model based on the offers the participants see on the screen and the choice outcome. E, Weights of logistic regression of choice from an extended model containing the standard model, RT, pupil rate, and other variables (see Results), showing a significant effort-by-pupil rate interaction. This extended regression (E) had a higher model-fit (adjusted R²) than the standard one (B; see Results), suggesting that pupil measures together with other task parameters can explain choice above and beyond the standard option attributes (reward and effort). Symbols indicate significance levels against zero: star for p < 0.001, triangle for p < 0.01, diamond for p < 0.05. Bar plots display mean ± 1 SEM. RT=response time; PBL=pupil baseline level.

During a fixation period of 3–6 s (drawn from a γ distribution with shape parameter 0.8 and scale parameter 1, mean 3.7 s), the text indicating reward and effort levels was masked with a series of letters X (Fig. 1A). Following this period, the color of the + sign at the center changed and the effort and reward of each of the two options were presented on either side of the fixation point for a fixed duration of 3 s. This prompted the subjects that they were able to press either the left or the right key to indicate their choice. To provide decision feedback, this key response was followed promptly by a change in color for the selected option. Regardless of key press, the stimuli remained on-screen for 3 s before the next fixation period was presented. If participants failed to respond during this period, the trial was coded as missing and no reward was gained. Among 49 participants, 13 had one missing trial, five had two to five missing trials, and one had 34 (15%) missing trials. Exclusion of this last subject did not change any result, so we decided to include them.

Pupillometry

Participants' right or left eye (depending on feasibility) was monitored using MR-compatible infrared EYElink 1000 eye-tracker system (SR Research Ltd.) with 500-Hz sampling rate. Participants were instructed not to blink during the presentation of the options. Preprocessing of the pupil data were performed in MATLAB (version 2017a, MathWorks). Data indicating eye blinks were replaced using linear interpolation. The data were visually inspected to ensure that all artefacts had been successfully removed. Pupil data were z-transformed within each run to control for variability across runs and across subjects. We focused on phasic pupil activity from stimulus onset until response, and therefore the time leading up to the choice during which participants focus on the choice-relevant information and use it to compute integrated decision values. Note that this measure therefore did not include time periods subsequent to choice during which participants may engage in decision outcome evaluation. Our prechoice pupil measure is called pupil rate of dilation (unit: z/s), and was calculated by subtracting pupil size at button response from pupil size at stimulus onset, divided by response time (RT). Pretrial pupil baseline level (PBL) was calculated by averaging pupil size from 500 to 1 ms before stimulus onset and served as an index of tonic arousal.

To ensure constant screen luminance level, we kept roughly the same number of pixels throughout the events by replacing the text indicating reward and effort levels with a series of Xs and by using text hues that were isoluminant to the gray background (RGB gray: 178.5, 178.5, 178.5; green: 50, 100, 10; purple: 118, 60, 206; blue: 53, 77, 229). Ensuring readability, we selected these hues out of 17 theoretically isoluminant hues where relative luminance was calculated as a linear combination of the red, green, and blue components based on the formula: Y = 0.2126 R + 0.7152 G + 0.0722 B. This formula follows the function that green light contributes the most to perceived intensity while blue contributes the least (https://www.w3.org/Graphics/Color/sRGB). Green was always fixed as the base hue and blue and purple were randomly assigned trial-by-trial to highlight the selected offer (Fig. 1A).

Additionally, in a control experiment, we recorded luminance-driven pupil dilation without any cognitive task during presentation of fixation screens with a series of Xs as fixation period and Ys to replace the text that would have indicated the effort and reward levels in the main experiment, each period lasting for 3 s. Participants were instructed to keep their eyes open but were not required to press any key. Just like in the main experiment, green was the base hue during fixation whereas blue and purple were used to highlight the text on one side of the screen. All stimuli were in the same text format as in the main task (Fig. 1). Order of hue and side assignment were all counterbalanced and pseudorandomized. We found no difference in mean pupil diameter during the presentation of these control stimuli in different hues, confirming that the pupil response in the main task was not driven by differences in text luminance.

fMRI acquisition and analysis

Functional imaging was performed on a Philips Achieva 3T whole-body MR scanner equipped with a 32-channel MR head coil. Each experimental run contained 225–244 volumes (voxel size, 3 × 3 × 3 mm³; 0.5-mm gap; matrix size, 80 × 78 (FoV: [240 140 (FH) 240]; TR/TE 2334/30 ms; flip angle, 90°; parallel imaging factor, 1.5; 40 slices acquired in ascending order for full coverage of the brain). We also acquired T1-weighted multislice gradient-echo B0 scans which were used for correction of deformations (voxel size, 3 × 3 × 3 mm³; 0.75-mm gap; matrix size, 80 × 80; TR/TE1/TE2 // 400/4.3/7.4 ms; flip angle, 44°; parallel imaging; 40 slices). Additionally, we acquired a high-resolution T1-weighted 3D fast-field echo structural scan used for image registration during postprocessing (170 sagittal slices; matrix size, 256 × 256; voxel size, 1 × 1 × 1 mm³; TR/TE/TI // 8.3/3.9/1098 ms).

We used Statistical Parametric Mapping (SPM12; Wellcome Trust Center for Neuroimaging, London, United Kingdom; http://www.fil.ion.ucl.ac.uk/spm) for imaging analyses. Five preprocessing steps included (1) realignment and unwarping, (2) slice-timing correction, (3) co-registration and normalization, (4) smoothing, and (5) correction for physiological noise. First, we re-aligned all functional volumes to the first volume to correct for inter-scan movement. Images were unwarped using field maps to remove unwanted variance because of field inhomogeneity (Anderson and Platten, 2011). Second, unwarped functional images were slice-time corrected (to the acquisition time of the middle slice). Third, each subjects' T1 image was co-registered (as reference image) with the mean functional image (as source image) using segmentation parameters performed on both images (Ashburner and Friston, 2004). These images were then normalized using the inverse deformation procedure and spatially re-sampled to 3-mm isotropic voxels. Fourth, all images were smoothed using a Gaussian kernel (full-width at half-maximum 8 mm). Finally, we performed correction for physiological noise via RETROICOR (Glover et al., 2000; Hutton et al., 2011) using Fourier expansions of different order for the estimated phases of cardiac pulsation (third order), respiration (fourth order), and cardio-respiratory interactions (first order; Hutton et al., 2011). We created the corresponding confound regressors using the PhysIO Toolbox (Kasper et al., 2009; https://www.translationalneuromodeling.org/tapas).

We performed random-effect, event-related statistical analyses. For each subject, we first computed a statistical general linear model (GLM) by convolving a series of boxcar functions (time-locked to the stimulus onsets and with the trial-wise RT as each event's duration) with the canonical hemodynamic response functions and their first derivatives (temporal derivative). We also added to these GLMs 18 physiological regressors and 6 motion parameters. At the second level, we then tested the significance of subject-specific effects (as tested by t-contrasts at the first level) across the population. For these analyses, we used a gray matter mask as an explicit mask, created by averaging across subjects and smoothing (8 mm) all participants' normalized gray matter images (wc1*.nii) from the “segment” procedure.

We built three first level GLMs without any orthogonalization. To identify unique variance associated with each of our trial parameters, we generated GLM1 using the stimulus onset as a single regressor with choice (1: effortful, −1: non-effortful), reward and effort levels of the effortful option, RT, pupil rate, and effort-by-choice as trial-wise parametric modulators (all non-binary variables were z-scored and interaction terms were constructed with z-scored variables). We then entered the contrast images of each parametric modulator versus baseline into second-level one-sample t tests. To illustrate the effort-by-choice interaction effect, we generated GLM2 with two regressors containing the stimulus onsets for “choose effortful” and “choose non-effortful” trials. Each regressor contained reward and effort levels of the effortful option, RT and pupil rate (all z-scored) as trial-wise parametric modulators. We created functional masks based on suprathreshold voxels for the effort-by-choice regressor from GLM1 and extracted β values within these masks. As control analysis for the results from GLM1, we built another first-level GLM (GLM3) containing the same regressors as GLM1 plus two additional regressors: reward-by-choice and RT-by-choice interaction terms. This analysis thus controlled for any differential effects of reaction time or reward across both choice types in the second-level analysis of our main interest, the choice-by-effort interaction.

Additional imaging tools. Functional masks were created using xjView toolbox (https://www.alivelearn.net/xjview/). SPM tables were created by extracting SPM results using bspmview toolbox (https://www.bobspunt.com/software/bspmview/). All suprathreshold clusters were overlaid on standard MNI structural brain image using MRIcroGL (https://www.nitrc.org/projects/mricrogl).

Experimental design and statistical analysis

We used a factorial design with six effort and six reward levels (36 cells) for the effortful option, and two reward levels for the non-effortful option. There were three trials in each cell, resulting in 6 × 6 × 2 × 3 = 216 trials. Trials were split in three fMRI runs of 72 trials (9 min) and trial order was pseudorandomized per subject and run.

Statistical analyses for behavioral and pupil data were done with MATLAB 2017 (MathWorks). We computed mean values per condition per subject for all our outcome variables except for RT, where we used median value to account for the fact that it is not normally distributed. We conducted (multiple) logistic or linear regressions separately for each participant and entered the regression weights of each predictor from all participants into a one-sample t test. All continuous predictors were z-scored across trials within each participant. This approach allows for the intercept (constant) to vary across participants. Goodness-of-fit is the adjusted R² for regressions. Effect sizes (Cohen's d) for one-sample t tests on the regression weights (Fig. 2E) were calculated as group-level mean of standardized regression weight for each regressor divided by its standard deviation. Effect sizes for paired-samples t test results related to figures 2 and 3 are reported as Cohen's d for repeated measures with pooled standard deviation calculated using the effect size calculator (https://www.psychometrica.de/effect_size.html#cohen; Lenhard and Lenhard, 2016). We used robust regression to evaluate the association between two variables. All statistical tests were two-tailed. For inference about the brain data, we used a cluster-defining threshold of p = 0.001 and only report suprathreshold voxels that survive cluster-level family-wise error (FWE) corrected p < 0.05. We also used this threshold to create functional masks for region of interest (ROI) analysis. Additionally, to create the left postcentral gyrus mask (Fig. 3F), we used a more stringent cluster-defining threshold of p < 0.00001 because the original suprathreshold cluster was too extensive (k > 4000). This was solely for illustration purposes and did not change data inference.

Figure 3.

Energization signals in pupil and dmPFC activity correlated with behavioral effort sensitivity. Consistent with the energization scenario, effort representations in pupil (A) and in the dmPFC (B) are higher when participants accepted compared with when they rejected the effortful option (choice “effortful” vs “non-effortful”). The positive modulation of effort-by-choice interaction is evidenced by higher effort β weights when participants chose the effortful (“yes” decision) versus the non-effortful option (“no” decision) in pupil (D) and in extracted BOLD signal change within dmPFC functional ROI (E). Both the pupil (G) and dmPFC (H) energization signals are positively correlated with individual behavioral measure of effort sensitivity as shown in Figure 1C. Consistent with neural processes that may reflect cost signaling, effort β weights in an extensive cluster with its peak in the postcentral gyrus (C, F) are higher when participants rejected compared with accepted the effortful option. However, the extracted cost signal within left postcentral gyrus functional ROI did not show evidence of any relation with the behavioral measure of effort sensitivity (I). A, Dots with error bars represent means ± 1 SEM. Lines are linear fits of the means [using the MATLAB polyfit(x,y,1) function]. B, Glass-brain image and sagittal slice showing that BOLD amplitude in dmPFC correlates uniquely with (positive) effort-by-choice regressor. C, Glass-brain image and coronal slice showing that BOLD amplitude in bilateral postcentral gyrus correlates uniquely with the (negative) effort-by-choice regressor. Panels E, F are solely for illustration purposes; no statistical test was done. Middle column, Boxplots display the median (central line), 25th and 75th percentiles (bottom and top edges), and non-outlier low and high extreme values (bottom and top error bars). Blue lines show subjects whose effort slope is higher in effortful choice than in non-effortful choice, gray lines show subjects who show the opposite effect. Symbols indicate significance levels between two conditions: star for p < 0.001, triangle for p < 0.01, diamond for p < 0.05. Right column, Each data point represents a subject; p values represent significance level from robust regressions.

To calculate the effort slope in all our analyses (Fig. 3), for each subject we first averaged the pupil rate (z-scored within subject) in each of the six effort levels separately for trials where subjects chose the effortful option and those where they chose the non-effortful option. We then ran a simple regression of the averaged pupil rate on effort levels (levels 4–9), separately for each choice outcome. Without any missing data, the effort slope in each condition should be estimated based on six data points. However, choice was clearly affected by effort level (Fig. 1C), thus one concern is that for some subjects, there might have been too many empty cells (e.g., if options with effort levels 7–9 were never selected by a participant). If this were the case then there would be unequal number of data points to estimate the effort slope in one choice outcome versus another. To address this concern, we found that on average there were more than five pairs of data points in both choice outcomes (M_{non-effortful} = 5.59, SD = 0.67, M_effortful = 5.61, SD = 0.7), and importantly there was no significant difference between the two choice outcomes, t₍₄₈₎ = 0.13, p = 0.89. This result assured us that the estimation of effort slopes between the two choice outcomes was comparable.

Computational modeling of choice

To be able to quantify choice difficulty, we modeled how effort discounts the value of the offers by fitting a standard effort-discounting model (containing a valuation and a selection rule) to the choice data (Glimcher, 2009). This model contains a valuation rule assuming that the subjective value of an offer is determined as its associated reward devalued by its associated effort (Eq. 1). We selected a parabolic-discounting model based on previous physical-effort studies (Hartmann et al., 2013; Chong et al., 2017; Lockwood et al., 2017; Morel et al., 2017). The computed SV for each offer is entered into a selection rule that applies a softmax function to the difference of the subjective value, normalized by a choice temperature (Eq. 2). This selection rule also allows the sigmoid to be shifted horizontally by a bias term, to capture our observation that there is a constant bias toward accepting an effortful option (Eq. 2), as observed in the positive intercept in the logistic regression of choice (Fig. 2B). Here, are the equations for subjective value (1) and choice probability (2):

$$\mathrm{S}\mathrm{V}=\mathrm{R}-\mathrm{k}\mathrm{*}{\mathrm{E}}^2$$ (1)

$$\mathrm{p}\left(\mathrm{c}\mathrm{h}\mathrm{o}\mathrm{o}\mathrm{s}\mathrm{e}1\right)=\frac{1}{1+{\mathrm{e}}^{\mathrm{\beta }\mathrm{*}\left(\mathrm{d}\mathrm{e}\mathrm{l}\mathrm{t}\mathrm{a}\mathrm{S}\mathrm{V}\right)+\mathrm{b}\mathrm{i}\mathrm{a}\mathrm{s}}}$$ (2)

where SV = subjective value of an offer; R = reward amount of an offer, E = effort amount of an offer, k = effort discounting parameter, β = choice temperature, bias = constant toward accepting an effortful option.

We fitted the model to the data from each subject using the VBA toolbox (https://mbb-team.github.io/VBA-toolbox/) and analyzed the model posterior estimates for each subject. As expected, effort discounting “k” and the bias terms are significantly correlated with the choice proportion for the effortful option, robust regression, b_k(47) = −1.91, p = 0.045; b_bias(47) = 1.94, p = 0.026. The subject-wise posterior estimates demonstrated a significant bias (>0) at the group level, one-sample test, t₍₄₈₎ = 8.06, p < 0.0001, suggesting that overall, participants do exhibit a tendency to choose the effortful option regardless of the offered reward and effort amounts. Using this model, we were able to derive measures that allowed us to address two potential confounds in control analyses, namely the effects of choice difficulty and choice bias on pupil dilation. We quantified choice difficulty by calculating for each trial the absolute distance between SV (dSV) of the effortful and non-effortful options. Regardless of the final choice outcome, a larger distance means that the decision was easy (and vice versa).

Analysis plan

First, we explored whether prechoice pupil-linked arousal, as measured in rate of pupil change (Joshi et al., 2016; Reimer et al., 2016), scales monotonically with increasing effort, and, importantly, if such effort coding in the pupil rate is stronger for reject > accept choice outcomes (consistent with cost scenario) or for accept > reject choice outcomes (consistent with energization scenario). Studies in humans have mainly focused on absolute pupil size (de Gee et al., 2014), but we chose pupil rate specifically to control for effects on RT that may co-occur with pupil size changes as a consequence of arousal-related processes. Indeed, findings from animal neurophysiology have established clear precedents for using pupil rate in our study with humans; seminal work in monkeys by Joshi et al. (2016) reported that the fastest rate of pupil dilation occurred always ∼300 ms after LC firing, suggesting a tight relationship between LC firing and not just pupil size but also the speed of dilation. These studies underscore the importance of investigating pupil rate in humans and suggest that pupil activity may reflect the processing of important decision parameters that might at the same time, determine the duration of the engagement of neural choice processes. Therefore, it is important to be able study pupil signaling with measures that explicitly control for reaction time. In any case, note that we could replicate all the behavioral effects reported here in parallel analyses of absolute pupil size (pupil at response minus pupil at stimulus), showing that the pupil-related effects we report here are independent of the exact phasic pupil measure used.

Second, at the neural level, we similarly examined whether known cortical representations of effort reflect a neural version of such choice-dependent effort signal. Based on previous work with a similar paradigm (Prévost et al., 2010; Kurniawan et al., 2013; Meyniel et al., 2013; Skvortsova et al., 2014; Hauser et al., 2017), we expected these signals to be localized within the frontoinsular network, which based on its connectivity to the LC (Poe et al., 2020) may be strongly affected by NA arousal processes.

Third, if such effort signaling is at all behaviorally relevant, then we expect individuals who show stronger choice-dependent effort signals in pupil and the brain to display stronger effort sensitivity in their behavior, namely in choice frequencies. In the cost scenario, we would expect behavioral effort sensitivity to correlate positively with the difference in effort scaling of “no” > “yes” decisions, since individuals who assign higher costs to effort should forego the effort challenge more often. The energization scenario, by contrast, would predict behavioral effort sensitivity to be positively correlated with the difference in effort scaling of “yes” > “no” decisions, since those behaviorally more affected by effort would need a stronger energization signal to accept a given effort level.

Fourth, we conducted a series of control analyses to ascertain that the observed effects were not driven by changes in choice difficulty and reward value of the options. Moreover, since endogenous fluctuations of arousal states may cause a general bias toward exerting effort (Murphy et al., 2014), and since elevated emotional arousal before a force-production task can increase voluntary effort (Schmidt et al., 2009), we also controlled for effects of tonic pupil signals as indexed by pretrial PBL.

Data availability

Processed data and the code to reproduce the figures are available upon request.

Results

Systematic effort-reward trade-offs during choice

Initial analyses confirmed that participants indeed systematically traded off the proposed efforts and rewards when making decisions (Fig. 2A), as expected based on previous work (Kurniawan et al., 2010; Prévost et al., 2010; Chong et al., 2017). Effortful options were selected significantly more often when they offered higher rewards and lower effort amounts (logistic regression of choice; 1 = choose effortful, 0 = choose non-effortful; N = 49; adjusted R² M = 0.62, SEM = 0.017; t_reward(48) = 6.93, p < 0.0001; t_effort(48) = −7.25, p < 0.0001; Fig. 2B). In particular, effortful options were selected/abandoned most often when they were clearly attractive (high rewards for low effort)/unattractive (low rewards for high efforts), although the interaction effect was only marginally significant (t_{reward × effort(48)} = −1.93, p = 0.06). This “standard” logistic regression model confirms previous findings that decisions vary as a function of the offered rewards and the required effort. Furthermore, multiple regression of RT (median; z-scored) showed a significant effect of reward and choice (0 = non-effortful, 1 = effortful; N = 49; adjusted R² M = 0.148, SEM = 0.012), t_reward(48) = −5.13, p < 0.0001; t_choice(48) = −3.52, p = 0.0009 (other effects: t_constant(48) = 3.27, p = 0.002; t_effort(48) = 0.36, p = 0.72; t_{reward × effort(48)} = 12.27, p < 0.0001; Fig. 2D).

An energization signal in the rate of pupil change

We then investigated whether pupil change rate contained information correlated with choice outcome, over and above the known effects of reward and effort. To this end, we added pupil measures to the standard logistic regression of choice (Fig. 2B). This extended regression (Fig. 2E) replicated the effects of reward and effort (N = 49; adjusted R² M = 0.65, SEM = 0.018; t_reward(48) = 6.56, p < 0.0001, Cohen's d = 1.60; t_effort(48) = −7.39, p < 0.0001, Cohen's d = 1.91), and revealed a significant reward-by-effort interaction, t_{reward × effort(48)} = −2.41, p = 0.019, Cohen's d = 0.79. In this extended regression, we also accounted for variables including RT (t_RT(48) = −3.40, p = 0.0013, Cohen's d = 8.66), PBL (t_PBL(48) = 0.25, p = 0.80, Cohen's d = 0.25), and many others (t_{pupil_rate(48)} = −1.02, p = 0.31, Cohen's d = 0.68; t_{reward × PBL(48)} = 0.22, p = 0.82, Cohen's d = 0.22; t_{effort × PBL(48)} = −0.31, p = 0.75, Cohen's d = 0.32; t_{reward × effort × PBL(48)} = −0.61, p = 0.54, Cohen's d = 0.54; t_{reward × pupil_rate(48)} = −0.78, p = 0.44, Cohen's d = 0.55; t_{reward × effort × pupil_rate(48)} = 1.21, p = 0.23, Cohen's d = 0.83; t_constant(48) = 4.37, p = 0.0001, Cohen's d = 0.93). This extended regression had a higher model-fit (adjusted R²) than the standard regression that only contained reward, effort, and reward-by-effort, t₍₄₈₎ = 5.35, p < 0.0001, Cohen's d = 0.69, suggesting that pupil measures together with other task parameters such as reward, effort, and RT, can explain choice above and beyond the standard option attributes (reward and effort). Crucially, the extended regression revealed a significant interaction between effort level and pupil rate, t_{effort × pupil_rate(48)} = 2.04, p = 0.04, Cohen's d = 1.74.

To examine whether this interaction effect reflects stronger effort representations for “yes” > “no” choices (i.e., energization) or for “no” > “yes” choices (i.e., a cost signal, see Fig. 1A), we directly examined the slopes of the regressions of pupil signals on anticipated effort levels during both types of choice outcomes. Averaged across both types of outcomes, the regression slope was indeed positive (one-sample t test on averaged effort slopes across choice: t₍₄₈₎ = 3.24, p = 0.002) but importantly, it was significantly steeper when participants chose the effortful option compared with when they chose the non-effortful option, effort-by-choice interaction, t₍₄₈₎ = 2.59, p = 0.012, Cohen's d = 0.371 (Fig. 3C). Thus, the pattern of effort representations in pupil signal during “yes” and “no” choices is consistent with the scenario that arousal system engagement during choice relates to energization for the future challenge that is being pondered.

Neural responses in dorsomedial prefrontal cortex (dmPFC) also reflect energization

To identify neural processes that may similarly reflect energization, we then examined BOLD responses during the decision process. Analysis of the brain responses time-locked to the presentation of the options (stimulus onset) revealed a significant, and structurally similar, effort-by-choice interaction in dmPFC (positive modulation; covering both supplementary motor area (SMA) and anterior cingulate cortex (ACC); peak MNI space coordinates: [−3, 18, 45]; t value, 5.32; extent: 301 voxels; p < 0.0001 FWE; Fig. 3B, GLM1). No other brain areas showed signals that survived whole-brain FWE correction (Table 1). ROI analysis within the dmPFC functional cluster illustrates that the activity related to anticipated effort strength is indeed higher in trials where the effortful option was selected compared with foregone [“yes” > “no” decisions (Fig. 3D, GLM2 solely for illustration); see Table 1, effort-by-choice (positive modulation), for the relevant statistics]. Thus, similar to the pupil signals described above, BOLD activity in dmPFC also shows anticipatory effort signaling in a way that is consistent with energization to overcome future physical challenges. This effort-by-choice effect in the dmPFC remains significant (and reveals a highly overlapping suprathreshold cluster) in a control analysis where we simultaneously added regressors for reward-by-choice and RT-by-choice interaction (GLM3).

Table 1.

MNI coordinates and statistics for GLM1: effort-by-choice, choice, reward, effort, pupil rate, and RT modulation

Effect	Brain region	k	t value	p value	MNI coordinates
					x	y	z
Effort-by-choice (positive modulation)	L superior medial gyrus	301	5.324	<0.0001	−3	18	45
	L ACC		4.786		−6	27	27
Effort-by-choice (negative modulation)	L postcentral gyrus	4085	6.382	<0.0001	−33	−42	57
	R superior frontal gyrus		6.248		24	−6	66
	R postcentral gyrus		6.242		30	−42	57
	L middle temporal gyrus	147	4.688	0.009	−60	−30	−3
	L middle temporal gyrus		4.151		−57	−51	−6
Choice (effortful > non-effortful)	L middle frontal gyrus	146	4.749	0.02	−30	30	36
Choice (non-effortful > effortful)	No suprathreshold clusters
Reward (positive modulation)	L caudate nucleus	1320	7.417	<0.0001	−9	9	0
	R IFG (p. orbitalis)		6.752		36	21	−9
	L IFG (p. orbitalis)		5.662		−30	24	−3
	R middle frontal gyrus	464	7.237	<0.0001	39	21	27
	L inferior parietal lobule	1046	6.61	<0.0001	−36	−63	51
	L precuneus		4.949		−3	−66	42
	L middle occipital gyrus		4.948		−30	−78	27
	R middle temporal gyrus	211	5.767	0.003	60	−30	−6
	R middle temporal gyrus		3.781		57	−9	−18
	R fusiform gyrus	900	5.57	<0.0001	24	−81	−9
	L fusiform gyrus		5.044		−24	−78	−9
	L cerebelum (Crus 2)		5.018		−12	−81	−27
	L middle temporal gyrus	149	5.489	0.012	−60	−21	−15
	L superior frontal gyrus	478	5.06	<0.0001	−21	36	48
	L IFG (p. triangularis)		4.857		−39	21	24
	L middle frontal gyrus		4.756		−36	12	57
	L superior orbital gyrus	115	5.021	0.03	−30	51	3
	R inferior parietal lobule	597	4.922	<0.0001	33	−72	24
	R inferior parietal lobule		4.839		39	−60	48
	R middle temporal gyrus		4.706		54	−48	12
	L ACC	167	4.484	0.007	−6	42	12
Reward (negative modulation)	No suprathreshold clusters
Effort (positive modulation)	R rolandic operculum	151	5.503	0.008	48	3	12
	L linual gyrus	365	4.482	<0.0001	−15	−84	3
	L middle occipital gyrus		4.352		−30	−90	15
	L calcarine gyrus		4.073		3	−75	15
	R calcarine gyrus	117	4.313	0.022	30	−69	15
Effort (negative modulation)	No suprathreshold clusters
Pupil rate (positive modulation)	R precuneus	8334	7.993	<0.0001	6	−78	42
	L calcarine gyrus		7.485		−3	−90	0
	R superior frontal gyrus	1081	6.774	<0.0001	21	−9	69
	R middle frontal gyrus		6.639		45	−9	57
	L posterior-medial frontal		6.041		−18	−12	72
	R temporal pole	307	6.500	<0.0001	54	15	−6
	R insula lobe		4.806		36	9	12
	L rectal gyrus	287	5.715	<0.0001	−15	12	−9
	R caudate nucleus		4.519		9	6	3
	R olfactory cortex		4.220		21	9	−15
	L temporal pole	225	4.854	0.001	−57	9	−3
	L IFG (p. opercularis)		4.283		−42	9	15
	L IFG (p. orbitalis)		3.563		−30	27	−3
	R superior temporal gyrus	138	4.600	0.013	57	−39	27
Pupil rate (negative modulation)	L middle temporal gyrus	123	5.914	0.02	−51	−36	−3
	L inferior temporal gyrus		3.581		−60	−12	−21
	L angular gyrus	196	5.628	0.003	−54	−63	36
	L superior medial gyrus	280	5.168	<0.0001	−6	48	45
	R superior medial gyrus		4.652		3	30	60
	L middle frontal gyrus	96	4.703	0.044	−33	24	51
RT (positive modulation)	L cuneus	273	7.335	0.001	3	−93	24
	R cuneus		6.818		6	−84	42
	R paracentral lobule		6.657		6	−48	75
	R IFG (p. orbitalis)	219	5.795	0.003	45	36	0
RT (negative modulation)	R fusiform gyrus	11871	−14.006	<0.0001	36	−75	−9
	L fusiform gyrus		−12.751		−36	−69	−9
	L middle occipital gyrus		−11.346		−36	−87	6

All effects are from t tests; p values are at cluster-level FWE correction.

Energization signals in pupil and dmPFC relate to behavioral effort sensitivity

To investigate whether the energization signals in pupil and dmPFC activity are indeed behaviorally relevant, we tested whether the difference in effort coding (slope across effort levels) between “yes” and “no” responses was associated with individual differences in how the anticipated degree of effort affected choice outcomes. For this analysis, we performed for each individual a simple logistic regression of choice on the associated effort levels (transformed such that a positive slope means higher likelihood to forego the option with increasing effort). The individual slopes of these regressions, our behavioral measure of effort sensitivity, were indeed positively correlated with the strength of each individual's effort-by-choice effect in both pupil rate and dmPFC activity (taken from the ROI analysis), robust regressions b_{pupil_rate(47)} = 0.70, p = 0.043; b_dmPFC(47) = 3.56, p = 0.038; Fig. 3E,F). This effect does not seem to be driven by the precise form of the effort-sensitivity measure, as we found both energization signals to be positively related also with the computationally-derived parabolic effort discounting term (see computational modeling; robust regressions, b_{pupil_rate(47)} = 0.036, p = 0.07; b_dmPFC(47) = 0.304, p = 0.001). Thus, subjects with higher effort sensitivity (whose overall choice was more strongly affected by increasing effort) indeed showed, in both pupil rate and dmPFC activity, steeper effort coding when the effortful option was selected compared with when it was foregone. Therefore, the energization responses in pupil rate and the brain indeed appear to be relevant for guiding choices.

Consistent with neural processes that may reflect cost signals (stronger effort coding in “no” > “yes” choice outcomes), we found extensive suprathreshold clusters for a negative modulation of the effort-by-choice interaction, with the peak activation in the bilateral postcentral gyrus (Table 1). However, the cost signals extracted from this postcentral gyrus region do not show evidence of any relation with the behavioral measure of effort sensitivity, b_{postcentral(47)} = −0.033, p = 0.166. These findings suggest that both accounts of effort signaling (energization and cost) fit brain activity patterns in different brain regions, but in our dataset, only the energization signals in dmPFC relate to effort-based choice behavior.

Energization effects in pupil rate are independent of reward, net value, decision difficulty, tonic arousal, or choosing against the default option

A number of other effects in the pupil signal might be confounded with the energization signal in pupil rate. Here, we describe a series of control analyses that address each of the potential confounds one by one (Fig. 4). First, it is theoretically possible that the effects we observed in pupil rate reflect differences in reward or difficulty level of trials where effort was accepted versus rejected. Indeed, increases in pupil size have been observed for rewarding stimuli (Schneider et al., 2018) and trials that require greater cognitive control (van der Wel and van Steenbergen, 2018). In some cases, high-effort trials may indeed be associated with high rewards, hence making the decision to either select or forego the effortful option more difficult. Our behavioral results had already contradicted these alternative explanations, since they were derived with statistical models that accounted for any variance associated with reward levels and RT (an indirect proxy for decision difficulty; Kiani et al., 2014). Nevertheless, to show more directly that the energization effect is clearly independent of reward and difficulty, we repeated the pupil analyses depicted in Figure 3 but now on the residuals of pupil rate after partialing out the effect of reward, RT, and choice difficulty (dSV; see computational modeling; orthogonalization of pupil rate relative to these variables, one at a time). Once again, these control analyses revealed the effects already shown in top row of Figure 3, namely, (1) stronger effort signals in residual pupil rate when participants accepted versus rejected the effortful option; t_{resid_reward_out(48)} = 2.59, p = 0.012; t_{resid_RT_out(48)} = 2.53, p = 0.014; t_{resid_dSV_out(48)} = 2.56, p = 0.013; and (2) significantly positive associations between the pupil energization effect (effortful > non-effortful) and the behavioral effort-sensitivity parameter (robust regression b_{resid_reward_out(47)} = 0.71, p = 0.043; b_{resid_RT_out(47)} = 0.68, p = 0.048; b_{resid_dSV_out(47)} = 0.71, p = 0.043; Fig. 4A–C).

Figure 4.

The energization signal in pupil rate is independent of value and choice difficulty, relates only to effort (not reward), and is only evident in phasic (not tonic) arousal. A–C, Control analyses replicating the pupil-rate effects reported in Figure 3 while regressing out (one at a time) the effect of reward (A), RT (B), and choice difficulty (denoted dSV; C). These analyses establish that the effort-by-choice pupil-rate effects are not confounded by reward and choice difficulty. Panels D, E show selectivity analyses to rule out reward effects on pupil rate (D) and effort-by-choice interaction effects on tonic arousal (pupil baseline; E). The absence of effects in the selectivity analysis confirms the specificity of our findings for effort-by-choice signals and phasic arousal as measured in pupil rate. First column, Dots with error bars represent means ± 1 SEM. Lines are linear fits of the means [using the MATLAB polyfit(x,y,1) function]. Middle column, Boxplots display the median (central line), 25th and 75th percentiles (bottom and top edges), and non-outlier low and high extreme values (bottom and top error bars). Blue lines show subjects whose effort slope is higher in effortful choice than in non-effortful choice, gray lines show subjects who show the opposite effect. Symbols indicate significance levels between two conditions: star for p < 0.001, triangle for p < 0.01, diamond for p < 0.05, ns for p > 0.05. Right column, Each data point represents a subject; p values represent significance level from robust regressions. rew=reward; dSV=absolute delta subjective value; resid= residual.

Furthermore, to rule out an alternative explanation that the pupil is merely coding for any option attribute that participants experienced as result of their choice (in our case the other option attribute was reward), we replaced these analyses with a reward-by-choice interaction (instead of effort-by-choice). To rule out the alternative explanation that pupil rate in this experiment could be simply signaling value, we tested for a reward-by-choice effect in pupil rate (Fig. 4D), revealing a non-significant choice difference (effortful vs non-effortful) of the reward slopes, t₍₄₈₎ = 0.22, p = 0.82. An analogous behavioral measure for reward sensitivity was not significantly associated with the reward-by-choice effect either, robust regression b₍₄₇₎ = 0.54, p = 0.22. We also ruled out that our results might reflect potential correlations of pupil signals with net value (reward subjectively subtracted by effort). Since SV is entirely confounded with choice, we could not run SV-by-choice control analysis (as we did with reward-by-choice). Instead, we calculated for each subject the Spearman's correlation between pupil rate and computationally derived SV of the effortful option. Across all subjects, the mean Spearman's correlations were not significantly different from zero, t₍₄₈₎ = 0.11, p = 0.91; M = −0.001, SD = 0.11. Thus, the energization effect we identified in the pupil rate is independent of reward value, net value, decision difficulty, or a reward-by-choice interaction. These results confirm that pupil rate indexes anticipated energization, by signaling effort amounts one has committed to accept. Moreover, our results suggest that these energization signals reflect different neural mechanisms than those underlying conflict-driven pupil dilations and behavioral adjustments (Ebitz and Platt, 2015).

To ascertain that our novel effect is independent of another aspect of the arousal system, namely ongoing tonic arousal, we tested how pretrial PBL may relate to choice behavior. First, we directly tested for an effort-by-choice effect in PBL (Fig. 4E), revealing a non-significant choice difference (effortful vs non-effortful) of the effort slopes in PBL, t₍₄₈₎ = 0.45, p = 0.65. The behavioral measure of effort sensitivity was not significantly associated with the effort-by-choice effect either, robust regression b₍₄₇₎ = 0.35, p = 0.39. These results confirm that the choice-modulated effort representations are primarily expressed in how fast the pupil dilates but not in endogenous pretrial pupil fluctuations. Second, we examined choice proportions as a function of PBL median/tertile/quartile splits. We did not find any choice differences across PBL bins, F_mediansplit = 0.004, p = 0.94; F_tertilesplit = 0.53, p = 0.58; F_{quartilesplit} = 1.13, p = 0.33 (Fig. 5A). We then ran a logistic regression of choice on PBL, RT, reward, effort, and the interactions. We found no effect of PBL or any interactions with PBL on choice, t_PBL(48) = 0.987, p = 0.32; t_{reward × PBL(48)} = 0.5, p = 0.82; t_{effort × PBL(48)} = −0.9, p = 0.36; t_{reward × effort × PBL(48)} = −1.12, p = 0.26; other effects: t_constant(48) = 4.31, p = 0.0001; t_RT(48) = −2.16, p = 0.035; t_reward(48) = 6.43, p < 0.0001; t_effort(48) = −7.91, p < 0.0001; t_{reward × effort(48)} = −2.66, p = 0.01; t_{reward × RT(48)} = −1.93, p = 0.05; t_{effort × RT(48)} = 4.18, p = 0.0001; t_{reward × effort × RT(48)} = 2.32, p = 0.02 (Fig. 5B). Next, we inspected whether regressing out influences of previous trial from PBL would improve regression of choice of the current trial. To do this, we first ran a linear regression of the current trial's PBL with reward, effort, choice, RT, and ITI of previous trial (t-1) as regressors. Then we took the residual variance of this regression and used it as a regressor together with RT, reward, effort, and reward-by-effort interaction to fit choice of current trial. This analysis shows no significant effect of the residual PBL (PBL*) on explaining choice on current trial, t_residPBL(48) = −0.02, p = 0.97, other effects: t_constant(48) = 25.91, p < 0.0001; t_RT(48) = −3.26, p = 0.002; t_reward(48) = 14.83, p < 0.0001; t_effort(48) = −16.77, p < 0.0001; t_{reward × effort(48)} = 2.61, p = 0.012 (Fig. 5C). Together, these control analyses fail to show any contribution of tonic arousal to effort choice, thus confirming the specificity of the energization effect for phasic arousal responses during the decision process (within-trial).

Figure 5.

No effects of endogenous arousal fluctuations on choice rate. A, Choice proportion for the effortful option as a function of pretrial PBL bins. B, Weights of logistic regression of choice on reward, effort, RT, PBL, and the interactions. C, Weights of logistic regression of choice on reward, effort, RT, and residual variance of PBL after regressing out influences from previous trial (t-1). Bar plots display mean ± 1 SEM. Symbols indicate significance levels against zero: star for p < 0.001, triangle for p < 0.01, diamond for p < 0.05. c, constant; RT, reaction time; R, reward levels; E, effort levels; PBL*, residual PBL.

Finally, another potential confound is the observation that the pupil dilates more when participants choose an option that goes against their baseline choice tendency (de Gee et al., 2014), here denoted as the choice-bias effect. To address this potential confound, we re-defined decisions as those that were either consistent (“for”) or inconsistent (“against”) with the majority of the decisions that participant made, and compared pupil rate for these two choice outcomes. Extending the effects reported by de Gee et al. (2014), we found significantly higher pupil rate when subjects chose an option that went against their bias than when they chose an option consistent with their bias, t₍₄₈₎ = 5.04, p < 0.0001 (Fig. 6A). We also replicated the effect using the measure originally used in de Gee's study (absolute pupil dilation), t₍₄₈₎ = 5.95, p < 0.0001; t₍₁₇₎ = 6.28, p < 0.0001. Taken together, these data show that the pupil dilates faster in trials where people were deciding against their overall bias.

Figure 6.

Energization signal in pupil rate is independent of choice bias effect. Control analysis extending the effects reported in de Gee et al. (2014) that pupil dilates faster when participants choose an option against their default bias, denoted as choice-bias effect (A). The energization effect depicted in Figure 3 is not related to the choice bias effect (B). A, Boxplots display the median (central line), 25th and 75th percentiles (bottom and top edges), and non-outlier low and high extreme values (bottom and top error bars). Blue lines show subjects whose effort slope is higher in effortful choice than in non-effortful choice, gray lines show subjects who show the opposite effect. Symbols indicate significance levels between two conditions: star for p < 0.001, triangle for p < 0.01, diamond for p < 0.05, ns for p > 0.05. B, Each data point represents a subject; p value represents significance level from robust regression.

To rule out that this choice-bias effect confounds the energization effect we report in the pupil rate, we correlated the computational bias term (Eq. 2) with the pupil energization effect [effort β (choose_effortful minus choose_non-effortful)]. This revealed no significant relation, b₍₄₇₎ = 0.73, p = 0.4, arguing for independence of the two effects. We also found that the size of the bias-related difference in pupil rate (comparing trials with choices “against” > “for” the bias) did not correlate significantly with the energization effect in the pupil rate, b₍₄₇₎ = −0.04, p = 0.84 (see Fig. 6B). Thus, both control analyses demonstrate that the energization effect in pupil rate is independent from the previously-reported choice-bias effects.

Energization effects in dmPFC are independent of neural representations of other task parameters

We also made sure that the observed energization effects in the dmPFC are indeed novel and separate from known neural correlates of effort-based decisions. To this end, we had included the effort-by-choice interaction as a regressor together with main effects of choice, reward, effort, pupil rate, and RT in the same model without any orthogonalization (see Materials and Methods). This allowed us to identify neural representations that are unique to each of the task parameters, ensuring that the effort-by-choice interaction cannot be explained by any combination of the other factors and allowing us to inspect our data for several known neural representations active during effort-based decisions (Fig. 7).

Figure 7.

Neural representations of choice, reward, effort, RT, and pupil rate. These plots show whole-brain statistical parametric maps for neural representations of choice (effortful > non-effortful), reward, effort, RT, and pupil, p < 0.05 FWE corrected. These established effects were derived with the same statistical model also used to identify the energization signals displayed in Figure 3 (GLM1); the latter signal is therefore specific and unrelated to these classic effects reported in the literature. Top right, Group-averaged bivariate Spearman's correlations between regressors in GLM1.

Consistent with previous demonstrations of the role of the dorsolateral prefrontal regions in executive function (Grueschow et al., 2020), we observed higher activity for choosing the effortful options compared with the non-effortful options in the left medial frontal gyrus. We also replicated previous findings of reward-related activity increases in the brain valuation system (Bartra et al., 2013; Burke et al., 2013), with peak activity in the ventral striatum, and effort-related activity decreases in the insula (Prévost et al., 2010). Moreover, we found slower button responses to be associated with higher activity in inferior frontal gyrus and faster responses to be associated with higher activity in a frontoparietal network that is often implicated in task engagement (Dosenbach et al., 2008; Cole et al., 2013). Finally, we found faster pupil rate to be associated with lower amplitudes of BOLD responses to the presentation of the stimuli in the middle temporal gyrus. By contrast, faster pupil rate was associated with higher BOLD amplitudes in a large-scale network within the occipital cortex (extending to precuneus), consistent with established involvement of this network in visual processing (Goodale and Milner, 1992). Thus, our brain results show that the energization signal in dmPFC is a conceptually new choice signal that is clearly distinct from previously observed effects of reward, effort, choice outcome, RT, and pupil signals (all reported effects survive whole-brain FWE correction; full statistics in Table 1).

Controlling for luminance-driven pupil response

Finally, to rule out brightness-induced pupil dilation and to validate our selection of theoretically isoluminant stimuli, we recorded pupil response during a control experiment at the end of the fMRI scan. Here, the same participants received similar visual stimulation as in the main experiment, but without informative cues or any need for making a choice. Participants were first presented with the same fixation screen (Fig. 2A, screen with XXX) with letters written in green for 3 s. This was followed by the same screen but with all Xs replaced by Ys, and in either one of the sides (counterbalanced), the letters were printed in either purple or blue ink (to mimic the visual change found in the main experiment) for another 3-s period. All three-color hues are theoretically isoluminant as described in Materials and Methods. There were 20 trials for each side and each hue, summing to 80 trials. We confirm that indeed the hue selection in a task without any reward-effort decision-making did not evoke meaningful luminance-driven pupil variance (Fig. 8). First, the scale of pupil response variance in the main task was at least 6 times larger than that in the control experiment. Second, if any, the deflection in pupil response to cue onset was negative, as opposed to that found in the main task. Third, this control experiment revealed no difference in averaged pupil size across the entire stimulus duration between the two isoluminant hues (purple and blue) used in the main task, paired-samples t test: t₍₄₆₎ = 0.29, p = 0.76 (two missing data). These results confirm that the pupil dilation observed in the main task was primarily driven by meaningful cognitive considerations provoked by the choice task, in this case by effort-reward tradeoffs, and not by task-irrelevant physical differences in the stimuli.

Figure 8.

Pupil during main versus control experiment. A, Pupil time course in main and in control task for stimuli in blue and purple ink, subtracted by PBL. Inset, (B) Zoomed-in pupil time course and (C) averaged pupil size across 3 s, showing no difference in pupil responses between blue and purple. Bar plots display mean ± 1 SEM.

Discussion

Consistent with the energization scenario, our results show that during deliberation, effort is represented by the peripheral (as measured via rate of pupil dilation) and central nervous systems (selectively in dmPFC) in a way that may relate not to a “cost” but rather to simulating the energization required for the action. Importantly, our results control for general arousal effects that could have been driven by value, difficulty, choice bias, or tonic arousal, thereby emphasizing the functional specificity of this energization signal.

Our results emphasize that phasic pupil-linked arousal is related to chosen effort amounts. This effect in pupil rate occurred in the period leading up to the choice during which participants focus on the choice-relevant information to compute decision values. Our results therefore suggest that for an extended period before the overt response, the brain appears to focus on stimulus aspects that are most relevant for the upcoming decision (here the effort that would need to be exerted in the future). Note that conceptually similar processes have been proposed in the value domain, where several studies have found results consistent with anticipatory processing of the value for the to-be chosen option (Shimojo et al., 2003; Van Slooten et al., 2018). The case is less clear-cut for our fMRI results, which do not have the same fast temporal precision as the pupil signal and therefore cannot be unambiguously attributed to the time periods before or after choice. However, since the pupil and fMRI effects show very similar patterns and both correlate with the same behavioral measure, it is very likely that the fMRI results in dmPFC similarly reflect anticipatory energization before the overt response. It could be interesting in future studies to provide more direct evidence for a role of dmPFC in prechoice energization, for example with time-resolved brain stimulation in the period leading up to the overt response [as done, e.g., by Murd et al. (2020) in the perceptual domain].

What neural mechanisms may lie at the heart of this link between pupil and choice? While the temporal sluggishness of the BOLD signal makes it difficult to provide a conclusive answer, we outline two possible causal mechanisms based on recent advances. First, simulating energization could trigger a “bottom-up” arousing influence that pushes decisions toward effort. This is consistent with the widely held view (Glimcher, 2009) that the strength of representations for decision attributes directly influence choice, for instance, it has been shown that intensifying encoded rewards through simulation of future events is linked with decisions that promote higher long-term pay-offs (Peters and Büchel, 2010; Benoit et al., 2011; Dassen et al., 2016; Bulley and Gullo, 2017) and increases prosocial behavior (Gaesser et al., 2018). Given this assumption, the arousal signal we observed might either down-modulate anticipated effort costs or shift the decision rule (de Gee et al., 2014), implying that a strong arousal signal could bias a decision toward accepting the physical challenge. One plausible bottom-up pathway includes feedforward transmission by phasic LC activity to ACC via ascending projections to prefrontal areas (Porrino and Goldman-Rakic, 1982; Chandler et al., 2013; Schwarz et al., 2015). Nervous-system readout of autonomous arousal could provide a signal for the organism that it is indeed ready to exert effort, instantiating an additional mechanism to bias choices.

Second, simulated energization could be a by-product of choice, implying a top-down influence from the cortical decision circuit to arousal. Decision outcomes could be relayed as descending input from the PFC into LC. Recent evidence in human fMRI demonstrates that dmPFC is functionally coupled with LC in a behaviorally-relevant way: the study showed that the strength of dmPFC-LC functional coupling is related to the level of individual cognitive control (Grueschow et al., 2020). Similarly, ACC/dmPFC activity has been found to be coupled with pupil diameter in non-human primates during cognitive control (Ebitz and Platt, 2015). Moreover, it has been shown that the timing of pupil modulation by ACC precedes that by LC (Joshi et al., 2016). Existing tracing data in rodents and monkeys also show afferent PFC projections as the main direct cortical influence on LC (Dalsass et al., 1981; Arnsten and Goldman-Rakic, 1984). Intracranial stimulation in human ACC leads to subjective accounts of changes in arousal states, coupled with the anticipation of challenges and a strong motivation to overcome obstacles (Parvizi et al., 2013). Taken together, these observations are consistent with the idea of a top-down influence from dmPFC to the NA arousal system (Aston-Jones and Cohen, 2005; Grueschow et al., 2020) that may serve to transmit information about the commitment to exert effort, thus resulting in speeded upregulation of arousal to prepare for the future challenge. Future studies may need to employ neuroimaging methods with higher temporal resolution to disambiguate these two hypotheses. Such studies may also employ pharmacological manipulation to increase NA tone activity, bio/neuro-feedback with pupil/LC activity, and mental simulation training (Steinmetz et al., 2018) to increase arousal in a bottom-up fashion.

In our study, future efforts were signaled by the pupil-linked arousal system and dmPFC activity during choices that preceded actual exertion by ∼1 h. Why would the brain energize in situations when it seems unnecessary? In line with the memory-based simulation account of decision-making (Wimmer and Shohamy, 2012; Bornstein and Norman, 2017), we believe that energization signals may not serve to immediately enhance motor readiness but rather to simulate, based on prior experiences, to what degree the organism would be able to take on this challenge in the future (similar to how anticipatory reward signals during such experiments may be used to simulate the pleasure derived from consuming this reward at a much later time point). That the noradrenergic system is capable of such simulation, and that this can be read out via pupil signals, is indicated by several previous studies. For example, it has been shown that despite constant illumination levels, the pupil responds to mere imagination of contextual settings that people have strong experiences with (e.g., mere recall of bright stimuli; Zokaei et al., 2019). We believe similar experience-related simulations may underlie the pupil effects observed here (Binda and Murray, 2015), indicating some form of unconscious simulation of the energization required to take on the challenge in the future.

Our results may also seem at odds with those of monkey studies employing LC electrophysiology and NA pharmacology, which clearly showed effort sensitivity in the NA-system only during force production, but not during cues just moments before the effort (Varazzani et al., 2015; Jahn et al., 2018). The differences between our results and these datasets may reflect the very different time-periods separating choices from effort execution. As discussed above, in a paradigm such as ours, the brain may need to perform a mental estimation of the amount of resources that will have to be mobilized to make the decision. This kind of simulation may not be needed, or may even be counterproductive, when decisions and exertions occur within seconds of one another. These methodological differences are not specific to our case but are rather a reflection of the state of the literature. Many monkey studies presented forced or choice cues that directly preceded actual exertions, whereas many human studies presented choice cues involving efforts that are delayed or even hypothetical. We clearly need studies that systematically investigate how the different time courses present in these experiments affect effort coding in the NA arousal system and throughout the brain.

Our results highlight that choices may be jointly guided by DA and NA systems for reward and effort processing, respectively. Previous effort studies have mainly reported a net value representation (reward discounted by effort) within the core brain valuation network (Prévost et al., 2010; Aridan et al., 2019) and in dmPFC (Prévost et al., 2010; Burke et al., 2013; Klein-Flügge et al., 2016; Chong et al., 2017; Arulpragasam et al., 2018; Bernacer et al., 2019). These fMRI results are consistent with animal data showing reduced willingness to choose a high-effort/high-reward option when dopamine is depleted (Salamone et al., 2007) and with the overarching dopaminergic role in motivational reward processing (Walton and Bouret, 2019). Our data concur with these studies, showing reward coding within the brain valuation network (Prévost et al., 2010; Aridan et al., 2019) and notably NA-linked pupil dilations and dmPFC brain representations for physical effort (Kurniawan et al., 2013; Meyniel et al., 2013; Skvortsova et al., 2014; Zénon et al., 2014; Varazzani et al., 2015). This potential “partnership” of DA-coding for reward and NA-coding for effort does not seem to concur with the classical (but possibly simplistic) view that DA-linked reward processing is discounted in a subtractive fashion by NA-linked effort cost representations. We emphasize that our behavioral data and some aspects of our neural results are in line with previous computational suggestions that an option is selected based on a reward-effort trade-off (Fig. 2). However, to our knowledge, prior work in humans has not examined how arousal-linked effort coding directly relates to choice. Here, we were able to scrutinize this functional role using concurrent pupil-fMRI in an effort discounting task. Our results suggest that NA may play a complementary function to DA. Future studies may build on our results to further characterize the interaction between DA and NA, using the pupil rate measure to quantify energization signals that guide human decision-making.

Variations in pupil-linked arousal states, such as locomotion and sleeping, are coupled with oscillatory state changes in brain networks (Takahashi et al., 2010) that are thought to result from noradrenergic innervation to the cortex (Schwarz et al., 2015). However, there are also observations that arousal may relate to movement during wakefulness and REM sleep, which is guided by cholinergic neuromodulatory projections from the basal forebrain to the cortex (Saper et al., 2010). This raises the concern whether we can truly draw the conclusions that our pupil-linked arousal effects originate from LC-NA neuromodulation alone. While we cannot fully rule out potential effects of cholinergic activity, a recent analysis with pupil activity and noradrenergic and cholinergic projections demonstrated that pupil rate in mice is more tightly linked with NA projections to the cortex, whereas activity in the cholinergic pathways more closely matched pupil size (Reimer et al., 2016). Relatedly, a recent pharmacological study using clonidine to upregulate NA signaling in humans shows increased tonic pupil diameter during task-free intervals (Gelbard-Sagiv et al., 2018), but unfortunately does not report task-related phasic pupil rate, or a comparison with cholinergic signaling. Thus, data from mice generally support the view that our effects in pupil rate may reflect phasic arousal variations that most likely originated from NA-LC activity, but more human studies are needed to replicate these findings.

Our results have relevance for the diagnosis and therapy of brain disorders with deficits in motivated behavior. The inability to commit to effort may bring about a cascade of clinical symptoms of apathy (Kurniawan et al., 2011; Husain and Roiser, 2018; Le Heron et al., 2018). Neurocomputational work has identified promising phenotyping approaches of motivation disorders; these reflect key involvement of the frontosubcortical circuitry and neuromodulatory systems including dopamine, serotonin, and noradrenaline (Meyniel et al., 2016; Pessiglione et al., 2018; Berwian et al., 2020). A specific role for noradrenaline is suggested by the finding that deficits in depression, namely fatigue and loss of energy, are inadequately treated by serotonergic antidepressants, but can significantly improve following administration of NA (and dopaminergic) agents (Nutt et al., 2007). This highlights the critical yet overlooked role of NA in motivation regulation (Moret and Briley, 2011). Chronic exercise in mice increases LC-NA derived neuropeptide galanin that later conferred stress resilience (Tillage et al., 2020), providing evidence of an adaptive role of NA-related energization signal. Our study contributes to this literature, by showing that the pupil-brain arousal system is sensitive to deliberations regarding future efforts. Future work should thus incorporate autonomic arousal and noradrenergic systems in quantitative models of motivation deficits (Pessiglione et al., 2018).