Roles of Ventromedial Prefrontal Cortex and Anterior Cingulate in Subjective Valuation of Prospective Effort

Abstract

The perceived effort level of an action shapes everyday decisions. Despite the importance of these perceptions for decision-making, the behavioral and neural representations of the subjective cost of effort are not well understood. While a number of studies have implicated anterior cingulate cortex (ACC) in decisions about effort/reward trade-offs, none have experimentally isolated effort valuation from reward and choice difficulty, a function that is commonly ascribed to this region. We used functional magnetic resonance imaging to monitor brain activity while human participants engaged in uncertain choices for prospective physical effort. Our task was designed to examine effort-based decision-making in the absence of reward and separated from choice difficulty—allowing us to investigate the brain’s role in effort valuation, independent of these other factors. Participants exhibited subjectivity in their decision-making, displaying increased sensitivity to changes in subjective effort as objective effort levels increased. Analysis of blood-oxygenation-level dependent activity revealed that the ventromedial prefrontal cortex (vmPFC) encoded the subjective valuation of prospective effort, and ACC activity was best described by choice difficulty. These results provide insight into the processes responsible for decision-making regarding effort, partly dissociating the roles of vmPFC and ACC in prospective valuation of effort and choice difficulty.

Introduction

Our decisions are shaped not only by rewards but also by the perceived level of physical effort required to obtain these rewards. Therefore the subjective perception of effort plays a critical role in driving choice—if the perceived effort of an action exceeds its subjective reward, the decision maker will optimally choose not to perform the action. Moreover, the perception of effort impacts everyday decisions in a variety of contexts ranging from job search (DellaVigna and Paserman 2005) and performing labor (Abeler et al. 2011; Augenblick et al. 2015), to participating in exercise and physical activity (Dishman 1991; Sniehotta et al. 2005).

Subjective value signals for appetitive and aversive stimuli (e.g., money, food, aversive foods a liquids) have been found in the ventromedial prefrontal cortex (vmPFC) for both decision and outcome values (for comprehensive reviews of this literature see Bartra et al. 2013; Clithero and Rangel 2014; O’Doherty 2014). This body of work implicates vmPFC in the computation of subjective value across a multitude of stimuli with both positive and negative valence. Rather than experimentally separating such rewarding stimuli from effort costs, there have been a number of studies in animals (Walton et al. 2002, 2009; Floresco et al. 2008; Rudebeck et al. 2008; Hillman and Bilkey 2012) and humans (Croxson et al. 2009; Kurniawan et al. 2010, 2013; Prévost et al. 2010; Skvortsova et al. 2014; Bonnelle et al. 2016; Klein-Flügge et al. 2016; Chong et al. 2017) that examine how the brain makes decisions to trade effort for reward. This work suggests that anterior cingulate cortex (ACC) encodes the valuation of effort costs and effort cost trade-offs, both at the time of decision and at the time of effort exertion (Supplementary Table 1 summarizes behavioral paradigms and findings of previous studies of effort-based decision-making).

However, a recent series of studies, investigating neuroeconomic choice for prospective rewards, has shown that activity in ACC is best described by choice difficulty rather than valuation of the options presented (Shenhav et al. 2014, 2016). These experiments examined decisions regarding foraging costs and were careful to design their tasks such that prospective values were orthogonal to choice difficulty (i.e., the proximity in value between alternatives). From these studies, it has been suggested that brain activity related to executive processing (e.g., choice difficulty) could confound valuation signals in ACC and throughout the brain in a variety of experimental contexts (Hayden and Heilbronner 2014; Shenhav et al. 2014; Westbrook and Braver 2015; Ebitz and Hayden 2016; Kolling, Wittmann et al. 2016).

Thus, there are two potential limitations to the interpretation of previous work implicating a role for ACC in encoding effort costs: 1) effort costs were not experimentally isolated from reward and 2) effort costs may be correlated with choice difficulty. While a key feature of previous effort-based choice paradigms was that they examined decisions between effort and reward, they were not designed to experimentally isolate subjective effort costs from monetary or other reward-based incentives. When requiring participants to trade effort for reward, it is possible that individuals could use a strategy in which they transfer effort values under consideration into a monetary scale in order to facilitate decisions between effort and reward. As such, it would be difficult to disentangle whether the neural value signals observed in these studies are truly related to effort valuation alone, effort values translated into a monetary scale, or multiplexed effort/rewards signals. Moreover, these previous studies of effort-based decision-making were not specifically designed to orthogonalize choice difficulty and effort value, which leads to the possibility that the ACC activity reported in these works could be related to the cognitive control associated with choice difficulty regarding decisions about effort and not effort valuation per se. In this study, we attempted to address these limitations in the understanding of effort-based decision-making.

Here, we investigated the behavioral representations of subjective effort cost and how these effort preferences are encoded in the brain’s valuation and decision-making circuitry. We had participants perform a novel effort choice task while their neural activity was recorded with functional magnetic resonance imaging (fMRI). Our task did not involve monetary earnings, which allowed us to experimentally isolate effort cost (i.e., choices did not involve a trade-off between effort and reward). Furthermore, the effort choices presented were designed to experimentally separate choice difficulty and effort value to better understand the specific role ACC plays in effort-based decision-making. We hypothesized that, in a similar fashion to monetary rewards (Kahneman and Tversky 1979; Holt and Laury 2002), participants would exhibit subjectivity in their decisions for prospective effort. That is, effort representations would differ from the objective amount of effort and would instead contain a degree of subjectivity driven by an individual’s perception of how effortful a task feels. Since our paradigm was designed to isolate subjective effort costs, independent of reward, and choice difficulty, we hypothesized that behavioral representations of this effort subjectivity would be encoded in vmPFC—consistent with findings in human and animal studies investigating subjective value of appetitive and aversive stimuli. Additionally, we hypothesized that when choice difficulty and effort value were experimentally isolated, ACC would encode the former in concert with previous studies of neuroeconomic choice.

Materials and Methods

Experimental Setup

Presentation of visual stimulus and acquisition of behavioral data were accomplished using custom MATLAB (http://www.mathworks.com) scripts implementing the PsychToolBox libraries (Brainard 1997). During fMRI, visual feedback was presented via a projector positioned at the back of the room. Participants viewed a reflection of the projector in a mirror attached to the scanner head coil.

An MRI-compatible hand clench dynamometer (TSD121B-MRI, BIOPAC Systems, Inc.) was used to record grip force effort. During experiments, signals from this sensor were sent to our custom designed software for visual real-time feedback of participants’ effort exertion. Effort exertion was performed while participants held the force transducer in their right hand with arm extended while lying in the supine position.

To record participants’ choices, we used an MRI-compatible multiple button-press response box (Cedrus RB-830, Cedrus Corp.).

Experiment Procedures

Participants

All participants were right handed and were prescreened to exclude those with prior history of neurological or psychiatric illness. The Johns Hopkins School of Medicine Institutional Review Board approved this study, and all participants gave informed consent.

A total of 42 healthy participants took part in the experiment, and 8 of them were excluded from the final analyses for one or a combination of behavioral reasons. First, participants were excluded if they were unable to generate salient associations between effort levels and applied effort (n = 4; r-squared value between reported effort and perfect reporting <0.5). Second, inconsistent decision-making precluded participants from subsequent analyses (n = 5; random or near random choices, characterized by a temperature parameter <0.001). The final analyses included n = 34 participants in total (mean age, 23 years; age range, 18–34 years; 15 females).

Effort Paradigm

Prior to the experiment, participants were informed that they would receive a fixed show-up fee of $30. It was made clear that this fee did not, in any way, depend on performance or behavior over the course of the experiment.

The experiment began with acquiring participants’ maximum voluntary contraction (MVC) by selecting the maximum force achieved over the course of 3 consecutive repetitions on the hand clench dynamometer. During these repetitions participants did not have knowledge about the subsequent experimental phases and were instructed and verbally encouraged to squeeze with their maximum force.

Next, participants performed an association phase in which they were trained to associate effort levels (defined relative to MVC) with the force they exerted against the hand dynamometer (Fig. 1A). Effort levels existed on a scale that ranged from 0 (corresponding to no exertion) to 100 (corresponding to a force equal to 80% of a participant’s MVC). A single training block consisted of 5 trials of training for each target level, where the target levels varied from 10 to 80 in increments of 10, and training blocks were presented in a randomized order. We did not perform association trials at the highest levels of effort to minimize the possibility that participants would become fatigued during this phase. A single trial of a training block began with the numeric display of the target effort level (2 s), followed by an effort task with visual feedback in the form of a black vertical bar, similar in design to a thermometer, which increased in white the harder participants gripped the dynamometer (4 s). The bottom and top of this effort gauge represented effort levels 0 and 100, respectively. Participants were instructed to reach the target zone (defined as ±5 effort levels of the target) as fast as possible and maintain their force within the target zone for as long as possible over the course of 4 s. Visual indication of the target zone was colored green if the effort produced was within the target zone and red otherwise. At the end of the effort exertion, if individuals were within the target zone for more than two-thirds of the total time (2.67 s) during squeezing, the trial was counted a success. These success criteria were meant to ensure that participants were exerting effort for a similar duration across all effort conditions. To minimize participants’ fatigue, a fixation cross (2–5 s) separated the trials within a training block and 60 s of rest were provided between training blocks.

Figure 1.

Experimental paradigm. (A) Association phase; participants were trained to associate numeric effort levels with force exerted on a hand clench dynamometer. Effort levels ranged from 0 (no force) to 100 (80% of maximum grip force). A training block consisted of 5 trials each at a series of target effort levels. Each trial began with presentation of the target, followed by an effortful grip with real-time visual feedback of the exerted force represented as a bar that increased in height with increased exertion. A green visual cue was also displayed, within which participants were instructed to maintain their exerted effort. Feedback of success or failure was provided at the end of each trial. (B) Recall phase; participants were instructed to fill a horizontal bar by gripping the transducer. On each trial, the full bar corresponded to a different target effort level that was unknown to participants. Successfully achieving the effort target resulted in the bar turning from red to green. Following exertion, participants used push buttons to move a cursor along a 0–100 number line to select the effort level they believed they had squeezed. No feedback was provided as to the accuracy of participants’ reported effort levels. (C) Choice phase; participants were presented a series of risky gambles which involved choosing between 2 options: exerting a low amount of effort with certainty (“sure”), or taking a gamble that could result in either a higher level of exertion or no exertion with equal probability (“flip”). Gambles were not realized following a choice. At the end of the choice phase, to ensure participants revealed their true preferences for effort, 10 choices were randomly selected and played out such that any effort required would need to be exerted before they completed the experiment.

Following the association phase, we performed a recall phase to test if participants successfully developed a behaviorally salient association between the effort levels and the actual effort exerted (Fig. 1B). Participants were tested on each of the previously trained effort levels (10–80, increments of 10), 6 times per level, presented in a random order. Each recall trial consisted of the display of a black horizontal bar that participants were instructed to completely fill by gripping the transducer—turning the force-feedback from red to green once the target effort level was reached. For the recall phase, the full bar did not correspond to effort level 100 as in the previous phase, but instead was representative of the target effort level being tested in a particular trial. Participants were instructed to reach the target zone as fast as possible, to maintain their produced force as long as possible, and to get a sense of what effort level they were gripping during exertion (4 s). Following this exertion, participants were presented a number line (from 0 to 100) and told to select the effort level they believed they had just gripped. Selection was accomplished by using 2 push buttons to move a cursor left and right along the number line, and a third button to enter their believed effort level. Participants had a limited amount of time to make this effort assessment (4 s), and if no effort level was selected within the allotted time the trial was considered missed. No feedback was given to participants as to the accuracy of their selection.

Finally, during the choice phase of the experiment we scanned participants’ brains with fMRI while they were presented with a series of effort gambles and the choices from these gambles were used to characterize how individuals subjectively valued effort (Fig. 1C). Prior to being presented with the effort gambles participants were told that 10 of their decisions would be selected at random at the end of the experiment, and that they would have to remain in the testing area until they achieved the exertions required. This was done to ensure that participants were properly incentivized on each trial. Importantly, effort choices and realization occurred in separate sessions, which allowed us to examine the neural signals associated with prospective effort valuation unaffected by physical fatigue. A single effort gamble consisted of choosing between 2 options shown on the screen under a time constraint (4 s): one option entailed exerting a low amount of force (S) with certainty (known as the “sure” option); whereas the other entailed taking a risk which could result in either high exertion (G) or no exertion, with equal probability (known as the “flip” option). The effort levels were presented using the 0–100 scale that participants were trained on during the association phase. Participants made their choices by pressing one of the 2 buttons on a handheld button-box with their right hand with either their first or second digits. Gambles were not resolved following choice and participants did not perform the squeeze task during this phase of the experiment. Effort gambles (170 in total) were presented consecutively in a randomized order. Participants were encouraged to make a choice on every trial, however there was no penalty for failing to make a decision within the 4-s time window (average percent of missed trials across subjects 1.8% ± 3.8). Failure to make a choice on time was logged as a missed trial and was not repeated.

At the end of the choice phase, the computer selected 10 of the trials at random to be implemented. The outcomes of the selected trials, and only those trials, were implemented. In this way, participants did not have to worry about spreading their effort exertion over all of their trials. Critically participants were instructed that the experiment would not be completed, and they were to remain in the testing area, until they achieved the exertions randomly implemented from the choice phase.

Our effort-based choice task has 3 properties that are important to stress. First, in our experimental design, we exploit the theoretical equivalence between risk preferences and subjective valuation in such a way that we can measure subjective valuation via the presentation of risky choices concerning effort, a widely accepted practice in economics and decision-neuroscience (Camerer et al. 2005; Rangel et al. 2008). Second, it is important to note that these effort choices did not involve monetary earnings or exertion at the time of choice, which allowed us to experimentally isolate prospective effort value representations from reward and fatigue. A number of decision-making studies have used similar paradigms, in which choice options are not realized at the time of choice, to study neural signals related to prospective valuation (Plassmann et al. 2007; Chib et al. 2012; Chong et al. 2017; Crockett et al. 2017). Third, effort choices were designed to span a range of potential effort values, capturing behavioral extremes of choice acceptance and rejection, centered at indifference. In doing so, this design ensures that choice difficulty, the magnitude of the relative value of effort options (behaviorally indexed by reaction time), is orthogonal to the difference in value between the options.

While our paradigm attempted to isolate effort valuation independent of appetitive values, it is important to mention that costs never truly exist in complete isolation from reward. In previous experiments investigating effort/reward trade-offs and value-based decision-making, trial-by-trial motivation was controlled by offering prospective reward in order to incentivize participants’ responses. In this study, trial-by-trial motivation was not controlled via monetary incentives, but instead participants’ choices were motivated by their avoidance of effortful exertion. In this way, the motivational aspects of this study are controlled by prospective effort rather than rewards or effort/reward trade-offs.

Effort Choice Values

The effort amounts were chosen to accommodate a range of effort sensitivity. For each trial, we denote the ratio η = G/S, of the worst possible outcome (choosing to gamble and having to exert the positive effort level) to the amount of effort in the sure option. We reasoned that participants would primarily exhibit increasing marginal utility for effort, and we therefore chose a range of η ϵ [1.75, 2.75]. In our gamble set, the force level associated with the sure option ranged from 5 to 35 in increments of 3.25 and were multiplied by the ratio η to generate 100 unique effort gambles, all with effort levels below 100. To span a broader range of η, additional gambles were designed in a similar method described above, except that the ratio of flip to sure (η) was halved and then multiplied by the sure values (η_1/2 ϵ [0.88, 1.38]). Thirty of these 100 additional effort gambles resulted in trivial (G,S) pairings with the flip values less than sure values (η < 1) and were thus excluded. The end result was 170 unique effort gambles with η ϵ [1.00, 2.75] (see Supplementary Materials for the full choice set and η values).

Monetary Choice Task (Prospect Theory Task)

To investigate if there was a relationship between subjective preferences for effort and monetary gains and losses (i.e., risk aversion, loss aversion), a subset of participants (n = 22) performed a binary forced-choice task for money outside of the scanner. In this task, independent of the effort choice experiment, participants made series of choices between a certain option involving a payout with 100% probability and a risky option involving gain and loss with equal probability. This exact paradigm has been used in a number of studies to elicit subjective preferences for monetary gains and losses (Sokol-Hessner et al. 2009, 2012; Frydman et al. 2011; Chib et al. 2014).

MRI Protocol

A 3 Tesla Philips Achieva Quasar X-series MRI scanner and radio frequency coil were used for all the MR scanning sessions. High-resolution structural images were collected using a standard MPRAGE pulse sequence, providing full brain coverage at a resolution of 1 mm × 1 mm × 1 mm. Functional images were collected at an angle of 30° from the anterior commissure–posterior commissure (AC–PC) axis, which reduced signal dropout in the orbitofrontal cortex (Deichmann et al. 2003). Forty-eight slices were acquired at a resolution of 3 mm × 3 mm × 2 mm, providing whole-brain coverage. An echo-planar imaging (FE EPI) pulse sequence was used (TR = 2800 ms, TE = 30 ms, FOV = 240, flip angle = 70°).

Data Analysis

Effort Choice Analysis

We used a 2-parameter model to estimate participants’ subjective effort cost functions. We assumed a participant’s cost function $V(x)$ for effort $x$ as a power function of the form:

$$V\left(x\right)=-{\left(-x\right)}^{\rho },x\le 0$$

In this definition of effort cost, the effort level $x$ is defined as negative, with the interpretation being that force production is perceived as a loss. The parameter ρ represents sensitivity to changes in subjective effort value as the effort level changes. ρ < 1 indicates decreasing sensitivity to changes in subjective effort cost as the effort level increases. ρ > 1 indicates increasing sensitivity to changes in subjective effort cost as the level increases. ρ = 1 indicates that subjective effort costs coincide with objective effort costs. Essentially, ρ captures an individual’s bias to choose the gamble or sure option, given options of equal expected value.

Representing the effort levels as prospective costs, and assuming participants combine probabilities and utilities linearly, the relative value between the 2 effort options (with the sure prospect as the reference) can be written as follows:

$$\mathrm{R}{\mathrm{V}}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}\left(G,S\right)=\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}\left(\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}\right)-\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}(\mathrm{g}\mathrm{a}\mathrm{m}\mathrm{b}\mathrm{l}\mathrm{e})$$

$$\mathrm{R}{\mathrm{V}}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}\left(G,S\right)=-{\left(-S\right)}^{\rho }-(-0.5{\left(-G\right)}^{\rho })$$

$$\mathrm{R}{\mathrm{V}}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}\left(G,S\right)=0.5{\left(-G\right)}^{\rho }-{(-S)}^{\rho }$$

where RV_sure denotes the difference in value between the 2 options, and both G < 0 and S < 0 for all trials.

We then assume that the probability that a participant chooses the sure option for the kth trial is given by the softmax function:

$$P_k\left(\mathrm{R}{\mathrm{V}}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}\left(G,S\right)\right)=1/[1+\exp \left(-\tau R{V}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}\left(G,S\right)\right)]$$

where τ is a temperature parameter representing the stochasticity of a participant’s choice (τ = 0 corresponds to random choice).

We used maximum likelihood to estimate parameters ρ and τ for each participant, using 170 trials of effort choices $(G,S)$ with a participant’s choice denoted by $y\mathit{ϵ}\{0,1\}$. Here, y = 1 indicates that the participant chose the sure option. This estimation was performed by maximizing the likelihood function separately for each participant:

$$\sum _{k=1}^{170}{y}_i\log \left(P_k\left(G,S\right)\right)+\left(1-y_i\right)\log (1-P_k\left(G,S\right))$$

Monetary Choice Analysis

A separate maximum likelihood procedure was used to estimate parameters for monetary reward in a similar manner described by Sokol-Hessner et al. (2009, 2012), Frydman et al. (2011), and Chib et al. (2012), estimating both risk and loss aversion parameters for each participant. We expressed participants’ utility function u for monetary values x as

$$u\left(x\right)=\{\begin{array}{c}{x}^{\alpha },x\ge 0\\ \lambda x^{\alpha },x<0\end{array}$$

In this formulation, λ represents the relative weighting of losses to gains, and α represents the degree of a participant’s risk aversion. Assuming that participants combine probabilities and utilities linearly, the expected utility of a mixed gamble can be written as U(G,L,S) = (0.5 G^α + 0.5 λL) − S^α, where G, L, S are the respective gain, loss, and sure options of the presented risky option.

The probability that a participant chooses the risky option for the kth trial is given by the softmax function:

$$P_k\left(U\left(G,L,S\right)\right)=1/[1+\exp \left(-\tau U(G,L,S)\right)]$$

where τ is a temperature parameter representing the stochasticity of a participant’s choice (τ = 0 corresponds to random choice).

The maximum likelihood procedure was accomplished using 140 gambles with participant response y ϵ {0,1}. Here, y = 1 indicates that the participant chose to make a gamble. The estimation was performed by maximizing the likelihood function:

$$\sum _{k=1}^{140}{y}_i\log \left(P_k\left(G,L,S\right)\right)+\left(1-y_i\right)\log (1-P_k\left(G,L,S\right))$$

Mean and standard deviation for estimates are as follows: risk aversion: α = 0.81 (0.30), loss aversion: λ = 1.69 (1.32), temperature parameter: τ = 1.91 (1.14). Of the 22 participants who performed both the effort and monetary gamble tasks, 4 were excluded from this monetary analysis on the basis of inconsistent choices during the monetary task (random or near random choices, characterized by a temperature parameter <0.001) (n = 2) and parameter estimates beyond 2 standard deviations from the mean (n = 2). As the effort and monetary choice experiments occurred separately and were completely independent of one another, exclusion from the monetary experiment did not preclude an individual from being included in the main effort analysis.

Image Processing and fMRI Statistical Analysis

The SPM12 software package was used to analyze the fMRI data (Wellcome Trust Centre for Neuroimaging, Institute of Neurology; London, UK). A slice-timing correction was applied to the functional images to adjust for the fact that different slices within each image were acquired at slightly different time points. Images were corrected for participant motion by registering all images to the first image, spatially transformed to match a standard echo-planar imaging template brain, and smoothed using a 3D Gaussian kernel (8 mm FWHM) to account for anatomical differences between participants. Regressors modeling the head motion as derived from the affine part of the realignment procedure were included in the imaging models discussed below.

To examine regions of the brain that encode participants’ relative value of subjective effort costs, we estimated participant-specific (first level) general linear models (GLMs) for the effort choice phase of the experiment. This GLM included an event-based condition at the time of effort choice, and parametric modulators corresponding to both a participant’s difference in subjective value between the gamble and sure options RV_sure(G,S), and a behavioral measure for choice difficulty, log (response time). Trials with missing responses were modeled as a separate nuisance regressor. In addition, regressors modeling the head motion as derived from the affine part of the realignment procedure were included in the model. Using this model, we were able to test brain areas in which activity was related to participants’ difference in subjective value between the gamble and sure effort options and their underlying subjective effort value representations, as well as activity related to choice difficulty. Using these conditions, we created contrasts with the aforementioned parametric modulators, for difference in value, and choice difficulty at the time of effort choice.

To examine regions of the brain that encode participants’ chosen and unchosen effort values, we estimated another first-level GLM that included event-based categorical conditions corresponding to trials in which the gamble and sure options were chosen. Each of these categorical regressors included parametric modulators corresponding to the subjective utility ($V(x)$) of both the chosen and unchosen options for that trial. Trials with missing responses were modeled as a separate nuisance regressor. In addition, regressors modeling the head motion as derived from the affine part of the realignment procedure were included in the model. Using this model, we were able to investigate brain areas in which blood-oxygenation-level dependent (BOLD) activity was related to participants’ effort values for the chosen and unchosen options. Using these conditions, we created contrasts with the aforementioned parametric modulators for the chosen and unchosen effort values at the time of effort choice.

Statistical Inference

We analyzed the vmPFC signals shown within an independent region of interest (ROI) defined from an extensive meta-analysis of studies examining valuation of appetitive and aversive stimuli (5-mm-radius sphere centered at Montreal Neurological Institute coordinates [2, 46, −8]) (Bartra et al. 2013).

There is a degree of heterogeneity in the ACC activations reported in previous studies of effort-based decision-making. With this in mind, we analyzed the ACC signals shown within an independent ACC ROI taken at peak coordinates from Neurosynth.org (Gorgolewski et al. 2015) when using the term “effort” (5-mm-radius sphere centered at Montreal Neurological Institute coordinates [0, 14, 48]).

For these ROIs, we regressed our design matrix on a representative time course, calculated as the first eigenvariate. This provides a very sensitive analysis because only a single regression is performed for this region and no multiple comparisons are required. The results of these ROI analyses were used for all statistical inferences about brain activity and are reported in the main text.

To clarify the signal pattern in each ROI, we created plots of effect sizes within 3 equal-sized bins of the variable of interest (i.e., low, medium, high) at the peak of activity (Figs 3B and 5B). It is important to note that these signals are not statistically independent (Kriegeskorte et al. 2009) and these plots were not used for statistical inference. They are shown solely for illustrative purposes.

Figure 3.

vmPFC encodes subjective effort valuation. (A) A region of vmPFC in which BOLD activity was positively correlated with the relative value of the sure option at the time of choice, with peak activity at Montreal Neurological Institute (MNI) coordinates (x, y, z) = [−4, 46, −2]. The contrast shown in red was obtained at P < 0.005 (uncorrected) with a 10-voxel extent threshold. This contrast is significant at P < 0.05, small volume corrected in an independent vmPFC ROI. (B) BOLD effect size within a 5-mm sphere centered at peak activity in vmPFC was positively correlated with the difference in utility between the 2 options (RV_sure). This plot is not used for statistical inference (which was performed using an independent ROI analysis); it is shown solely to illustrate the trend of the BOLD signal in vmPFC. (C) Exceedance probability map (EPM) resulting from the Bayesian model comparison of objective and subjective effort valuation models. Voxels shown in green (n = 16) indicate locations in the brain where the probability that subjective effort describes the BOLD activity is greater than 0.90, supporting the finding that subjective effort cost best describes activity in vmPFC. (D) A region of vmPFC in which BOLD activity was negatively correlated with increasing subjective effort value of the chosen option ($V(x)$), with peak activity at MNI coordinates (x, y,z) = [−6, 46, −6]. The contrast shown in red was obtained at P < 0.005 (uncorrected) with a 10-voxel extent threshold. This contrast is significant at P < 0.05, small volume corrected in an independent vmPFC ROI. (E) BOLD effect size within our a priori vmPFC ROI for chosen V(chosen) and unchosen V(unchosen) effort value (*P < 0.05).

Figure 5.

ACC encodes choice difficulty. (A) Regions of the brain in which BOLD activity was positively correlated with choice difficulty at the time of choice with peak activity at MNI coordinates (x, y, z) = [8, 12, 50]. The contrast shown in blue was obtained at P < 0.005 (uncorrected) with a 10-voxel extent threshold. This contrast is significant at P < 0.05, small volume corrected in an independent ACC ROI. (B) BOLD effect size within a 5 mm sphere centered at peak activity in was positively correlated choice difficulty. This plot is not used for statistical inference (which was performed using an independent ROI analysis); it is shown solely to illustrate the trend of the BOLD signal in ACC. (C) BOLD effect size within our a priori ACC ROI for choice difficulty and RV_sure (***P < 0.001). (D) Average exceedance probabilities across all voxels within our a priori ACC ROI showed that choice difficulty was favored over RV_sure. This indicates a higher likelihood of ACC activity describing choice difficulty than RV_sure.

Bayesian Model Selection of Imaging Data

To determine if a subjective valuation of effort better accounted for neural activity in vmPFC than an objective representation, we performed a Bayesian model selection analysis (Rosa et al. 2010). We began by creating an additional GLM that was identical to our original model $\mathrm{R}{\mathrm{V}}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}\left(G,S\right)$, except in this model the parametric modulator corresponded to the difference in expected objective value of the effort options presented $\mathrm{R}{\mathrm{V}}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}^{\mathrm{o}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}}=0.5(-G)-(-S))$. This GLM captured the null choice model (objective valuation of effort; ρ = 1).

We used the first-level Bayesian estimation procedure in SPM12 to compute voxel-wise whole-brain log-model evidence maps for every participant and each model. To model inference at the group level, we applied a random-effects-like approach at every voxel of the log evidence data across the whole brain (Rosa et al. 2010). This random-effects approach allows for the possibility that different participants use different models for their underlying value computation. Log evidence data for each voxel were used to calculate posterior probability distributions of the frequency of each model given the observed group data. We used these data to create exceedance probability maps (EPMs) which allowed us to test which representation of effort cost, subjective or objective, was more likely to describe activity in vmPFC. The EPMs shown illustrate clusters of voxels at which subjective effort valuation, rather than objective valuation, has a greater Bayesian probability (P > 0.90) of describing the observed BOLD signal in vmPFC.

Given that we found both relative value and chosen value signals in vmPFC, in separate imaging models, we created additional log evidence maps for these models to evaluate which best described signals in vmPFC. For this analysis, we compared log evidence maps for the relative value of the 2 options $\mathrm{R}{\mathrm{V}}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}\left(G,S\right)$, the subjective value of the chosen option V(Chosen), and the difference in subjective value between the chosen and unchosen options V(Chosen)–V(Unchosen).

Finally, we performed a third Bayesian model comparison, this time investigating if relative difference in value between the effort options or choice difficulty was more likely to describe activity in ACC. From the whole-brain log evidence maps of each subject, comparing these 2 models, we were able to generate an EPM within the ACC ROI previously described.

Results

Behavioral Representations of Subjective Effort Valuation

Comparison between reported and exerted effort levels during the recall phase showed a high degree of agreement, indicating that participants accurately recalled the objective effort levels (Fig. 2A shows the group recall results; Supplementary Fig. 1 shows individual participant’s recall results). It is important to mention that this phase of the experiment was not meant to assess subjective valuation of effort but instead aimed to evaluate how well participants associate the units of effort levels to forceful exertion. The strong correlation between recalled and exerted effort levels indicates that participants understand the mapping between feelings of exertion and effort units. As a result, participants are able to make informed meaningful choices about prospective effort options that utilize this scale during the choice phase, and these choices are therefore not a by-product of uncertainty regarding units of effort.

Figure 2.

Behavioral representations of subjective effort cost. (A) Results from the recall phase of the experiment, showing the mean and standard error across all participants for the effort levels reported plotted against the those tested. The dashed line is included to indicate perfect recall of exerted effort. (B) The function used to model the subjective cost of effort in a choice. This function has the form $V\left(e\right)=-{(-e)}^{\rho }$. Each curve represents an individual’s cost function for effort. The dashed line is included to indicate an objective valuation of effort ($\rho =1$), with curves above this line representing that an incremental change in the effort level results in a greater subjective cost of that effort for higher effort levels. (C) Estimated $\rho$ parameters at the participant level. Asterisks indicate a significant difference (P < 0.05) from the null hypothesis of objective valuation ($\rho =1$) using a likelihood ratio test statistic. (D) Propensity to accept the sure option as a function of RV_sure. RV_sure was partitioned into 8 bins and the mean and standard error of the acceptance rate within each bin is displayed.

We characterized the subjectivity of participant i’s effort choices using a subjective cost function $V_i\left(x\right)=-{(-x)}^{{\rho }_i}$, where $x\le 0$ and $V$ is the subjective cost of an objective effort level $x$. ρ_i is a participant-specific parameter that characterizes how an individual subjectively represents the effort level $x$. In this formulation, ρ is flexible enough to capture increasing, decreasing, or constant marginal changes in subjective effort valuation as absolute effort levels increase (Fig. 2B). The case where ρ = 1 indicates that a participant’s subjective effort cost coincides with absolute effort levels, and that they exhibit no bias toward choosing risky or sure options when the prospects are of equal expected value. ρ < 1 indicates decreasing sensitivity to changes in subjective effort cost as the effort level increases, and that the participant exhibits a bias toward choosing the risky effort option when the prospects are of equal expected value. ρ > 1 indicates increasing sensitivity to changes in subjective effort cost as the effort level increases, and that participants have a bias toward choosing the sure effort option when the prospects are of equal expected value.

Using the behavioral data, we performed a maximum likelihood estimation procedure to characterize each participant’s subjectivity of effort cost ρ and underlying consistency of choice τ. We found that participants exhibited mean parameter estimates of ρ = 1.26 (S.D. 0.35), τ = 0.21 (S.D. 0.25). A parameter recovery procedure found a significant correlation between parameters initially estimated and those recovered, suggesting that the participants’ decisions over the effort choice options yielded a precise estimation of ρ (see Supplementary Material for details). A likelihood ratio test statistic indicated that the majority of participants (n = 23) made choices that were inconsistent with a linear subjective effort function (ρ = 1), and the group exhibited subjectivity parameters that were significantly >1 (t₃₃ = 20.76, P < 0.001) (Supplementary Table 3 includes parameter and significance estimates). These model parameters indicate that when the objective expected value of the certain and uncertain options are equal, participants display a bias toward choosing the sure option. It is important to note that the parameter estimates for τ, which characterize choice stochasticity, indicate that participants were neither making random or strictly rule-based decisions. We computed a group-level AIC using log-likelihood measures obtained from the MLE procedure and found AIC_objective = 5697 and AIC_subjective = 5215, further indicating that subjective valuation of effort best describes participants’ choices. We also performed a series of analyses using a variety of different utility functions, and found that the power function best described the choice data in our experiment (Supplementary Fig. 3). Together, these results reveal that participants did not make effort decisions purely based on an objective valuation of effort, and that the majority of participants instead exhibited subjectivity of effort such that larger effort levels yielded increased marginal effort costs (Fig. 2C). These behavioral findings are consistent with previous studies that modeled subjectivity of effort valuation when trading effort for reward (Klein-Flügge et al. 2015, 2016; Chong et al. 2017) and found that individuals had increasing marginal utility for effort as effort levels increased in an absolute sense.

We performed a series of analyses to determine if the subjective effort parameters were related to other potential factors that could influence effort valuation. We found that participants’ effort subjectivity parameters were not significantly correlated with MVC (r = −0.13, P = 0.45), suggesting that subjective preferences for effort were not simply the by-product of maximum strength. Additionally, for the individuals that completed both the effort and monetary choice tasks, effort subjectivity parameters did not correlate with measures of monetary subjective value (risk aversion: r = −0.21, P = 0.38; loss aversion: r = −0.17, P = 0.50), suggesting that individuals’ subjective preferences for reward are not related to their effort preferences and not simply a reflection of similar risk attitudes across decisions for different types of goods. Another possibility is that effort subjectivity parameters could be a reflection of the probability of success during the association phase—participants that were less successful at achieving the targeted exertions might find effort to be more costly and have higher ρ parameters. To test this possibility, we examined the relationship between ρ and success rate during the association phase. Again, we did not find a significant relationship between the two (r = 0.05, P = 0.78), suggesting that ρ is not driven by the probability of success during association (Supplementary Fig. 2B).

Next, we investigated the relationship between subjective effort valuation and decision difficulty. We found that participants’ decisions revealed that the difference in subjective utility between the two effort options $\mathrm{R}{\mathrm{V}}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}\left(G,S\right)=0.5{\left(-G\right)}^{\rho }-{\left(-S\right)}^{\rho }$ sampled the range of option rejection/acceptance (Fig. 2D). With this in mind, choice difficulty should be greatest when the effort options are most similar (when −|RV_sure| is near zero), and least difficult when the options are most dissimilar. Accordingly, response time will be longest when −|RV_sure| is small and shortest when choices are easiest. Consistent with this idea, we found that model-free (log(response time)) and model-based (−|RV_sure|) measures of choice difficulty were significantly positively correlated with one another (Pearson’s correlation coefficient range r ϵ [−0.21, 0.42]; t₃₃ = 2.21, P = 0.03; Wilcoxon signed-rank test on correlation coefficients: z = 2.03, P = 0.04). Furthermore, log(response time) was not correlated with the relative value between the 2 options (RV_sure and log(reaction time): Pearson’s correlation coefficient range r ϵ [−0.43, 0.26], t₃₃ = −0.98, P = 0.33; Wilcoxon signed-rank test on correlation coefficients: z = −0.97, P = 0.33). This orthogonalization between the subjective utility of effort and choice difficulty allowed us to identify the neural signals associated with each computational variable.

vmPFC Encodes Subjective Valuation of Effort

To test our neural hypothesis that subjective effort valuation is encoded in vmPFC, we estimated a GLM in SPM12 of the BOLD activity of the whole brain during the choice phase. This model included parametric modulators at the time of choice, corresponding to both the difference in value between the sure and gamble options RV_sure and choice difficulty as indexed by log(reaction time). RV_sure was defined by transforming the effort options under consideration using the effort subjectivity parameter ρ estimated from each individual participant’s behavior (see Materials and Methods for details). This formulation allowed us to isolate brain regions that encoded subjective valuation of effort and choice difficulty at the time of decision.

We found that BOLD signal in vmPFC was significantly positively correlated with RV_sure (ROI analysis of vmPFC: t₃₃ = 2.22, P = 0.03; Fig. 3A,B). As RV_sure increased, vmPFC activity significantly increased suggesting that this region encoded the difference in subjective effort value of the 2 options. The areas of vmPFC identified largely overlapped those found in studies of appetitive and aversive valuation (Bartra et al. 2013; Clithero and Rangel 2014; O’Doherty 2014) and are consistent with previous studies that found relative value signals in vmPFC associated with the comparison between 2 options that drive eventual choice (Basten et al. 2010; Lim et al. 2011; Shenhav et al. 2016). Notably, a recent study found that the sign of such an unsigned difference signal in vmPFC was modulated by attention (Lim et al. 2011). In these experiments, it was found that attended options resulted in stronger value signals in vmPFC, relative to the unattended option. Taking these results into account, our stronger value signal for the sure option might suggest that participants attended more to the sure option than the gamble option. However, we did not directly modulate attention in our experimental design, and a detailed examination of the sign of this relative value signal was beyond the scope of this study.

In a separate test, to confirm that activity in vmPFC during effort choices was best described by representing options subjectively as opposed to objectively (ρ = 1), we generated EPMs for the imaging model described above as well as a null model representing objective effort valuation ($\mathrm{R}{\mathrm{V}}_{\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}^{\mathrm{o}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}}=0.5(-G)-(-S)$) (Rosa et al. 2010). Using these probabilistic brain maps, we were able to evaluate the likelihood that areas of vmPFC better represented subjective effort costs as opposed to objective effort costs. We found a cluster of voxels in vmPFC, (6 voxels, P > 0.90), illustrating that activity in this region is best described by a subjective rather than objective model of effort costs (Fig. 3C).

To further dissect the components of effort choice, we created another GLM in which we modeled the subjective value of the chosen and unchosen options. Consistent with previous findings of value-based choice (O’Doherty 2011; Bartra et al. 2013; Clithero and Rangel 2014), we found significant activity in vmPFC associated with subjective utility of chosen effort value (Fig. 3D). An ROI analysis within our a priori ROI in vmPFC, showed a significant negative correlation with the utility of the chosen option (ROI analysis of vmPFC: t₃₃ = −2.60, P = 0.01), and significant differences between the subjective utility of the chosen and unchosen options (ROI analysis of vmPFC: t₃₃ = −2.49, P = 0.02). This finding is consistent with the idea that vmPFC activity decreases as the effort value of an option increases (i.e., its averseness or cost increases). Notably, even lowering our whole-brain significance level to a more liberal threshold (P < 0.05) did not reveal significant activity related to the chosen or unchosen value within ACC.

Given that we found overlapping regions of vmPFC encoding both RV_sure (Fig. 3A) and a chosen effort value signal (Fig. 3D), in separate imaging models, we performed additional analyses to determine which model of effort value best described activity in vmPFC. We first generated EPMs for RV_sure, chosen effort value V(Chosen), and V(Chosen)–V(Unchosen). Using these probabilistic brain maps, we found that V(Chosen) had the highest likelihood of describing vmPFC activity (Fig. 4). We also created a separate imaging GLM in which we included both relative RV_sure and V(Chosen), unorthogonalized, to allow them to compete for variance. In this analysis, we found significant clusters within our vmPFC ROI for both regressors (Supplementary Fig. 5). Together these results suggest that while vmPFC is best described by the chosen effort value, a degree of independent variance in this region is captured by both chosen effort value and relative effort value. These results are consistent with findings suggesting that vmPFC computes comparisons between options that result in an eventual decision (Basten et al. 2010; Lim et al. 2011; Bartra et al. 2013; Clithero and Rangel 2014; O’Doherty 2014; Shenhav et al. 2016).

Figure 4.

Model comparison for competing value models in vmPFC. Average exceedance probabilities across all voxels within our a priori vmPFC ROI showed that V(Chosen) was favored over RV_sure and V(Chosen) - V(Unchosen). This indicates a higher likelihood of V(Chosen) describing vmPFC activity than these alternative models.

To assess if the signals we found in vmPFC were simply a reflection of risk and not value per se, we performed a supplementary analysis in which we modeled neural activity using a mean–variance representation of risk. The analysis allowed us to separately examine brain regions related to objective effort value and variance/risk of the effort options. We found that vmPFC encoded the expected value of the effort options (Supplementary Fig. 4), and we failed to find brain activity related to the risk (variance) of the effort options at a very liberal threshold of P < 0.05, over the entire brain. This suggests that the value signals we find in vmPFC are not driven by neural representations of risk. These results align with previous studies of monetary risk preferences that found monetary values signals in vmPFC but did not find risk representations in the same region (Huettel et al. 2006; Preuschoff et al. 2006; Hsu et al. 2009; Suzuki et al. 2016).

ACC Encodes Choice Difficulty

To test our hypothesis that choice difficulty is encoded in ACC, we used the previously described imaging model to identify brain activity that was correlated with increasing choice difficulty. We found that choice difficulty was significantly positively correlated with activity in ACC (ROI analysis of ACC: t₃₃ = 6.03, P = 8.67 × 10⁻⁷; Fig. 5A,B), consistent with previous studies that have shown that this region encodes choice difficulty during neuroeconomic choice (Shenhav et al. 2014, 2016).

We also searched for effort value signals in ACC, as have been reported in previous studies, however even decreasing our contrast significance level to a more liberal whole-brain threshold of P < 0.01 did not reveal any significant activation in our ACC effort ROI. To directly test whether ACC activity was better described by choice difficulty or RV_sure, we performed a formal ROI analysis of ACC in which we compared the choice difficulty and RV_sure regressors and found that activity in ACC was significantly greater for choice difficulty than RV_sure (Fig. 5C, ROI analysis of ACC: t₃₃ = 5.98, P < 0.01). To further confirm that activity in ACC was best described by choice difficulty as opposed to RV_sure, we generated an additional EPM incorporating both choice difficulty and difference of relative effort value RV_sure. This analysis revealed that activity in ACC was better described by choice difficulty rather than RV_sure (ROI analysis of ACC: average probability across all voxels P = 0.98). This analysis indicates a 98% probability that choice difficulty is a better descriptor of the observed ACC activity than relative effort value, and a 2% probability that RV_sure is a better descriptor than choice difficulty.

It should be noted that in the aforementioned analyses we used a noisy model-free representation of choice difficulty: log(response time). We also performed a set of analyses in which we used model-based measures of choice difficulty (Fig. 6). These include a model that represented choice difficulty as the negative absolute difference in the relative value between the gamble and sure options −|RV_sure|, and another model in which |RV_obj| was normalized by the effort stakes $(-|\mathrm{R}{\mathrm{V}}_{\mathrm{o}\mathrm{b}\mathrm{j}}|/(0.5(0.5F+S))$. This stakes-normalized metric was used to account for the possibility that choice difficulty could be influenced by the magnitude of the effort options presented (analogous to the exponential increase in effort cost with increasing effort, captured by the effort utility function). These model-based choice difficulty analyses replicated our original finding of choice difficulty signals in ACC and provide further evidence that ACC activity represents choice difficulty in the context of effort-based decisions.

Figure 6.

Model-based measures of choice difficulty in ACC. (A) ACC BOLD signal is associated with a stakes-normalized choice difficulty measure. This measure is estimated as the negative absolute difference between the gamble and sure options, normalized by the effort stakes $(-|\mathrm{R}{\mathrm{V}}_{\mathrm{o}\mathrm{b}\mathrm{j}}|/(0.5(0.5F+S))$). This metric accounts for the possibility that choice difficulty could be influenced by the magnitude of the effort options presented (analogous to the experimental increase in effort cost with increasing effort, captured by the effort utility function). The contrast shown in blue was obtained at P < 0.005 (uncorrected) with a 10-voxel extent threshold. (B) ACC BOLD signal is associated with −|RV_sure|. Choice difficulty is estimated as the negative absolute difference between the gamble and sure options (−|RV_sure|). This model also included log(RT) to account for behavioral noise captured by response time. The contrast shown in blue was obtained at P < 0.005 (uncorrected) with a 10-voxel extent threshold.

Discussion

We used a novel effort choice paradigm in which participants were presented with prospective effort options under uncertainty, which were independent of reward and orthogonal to choice difficulty. This paradigm allowed us to isolate neural signals related to subjective valuation of effort that were not contingent on reward or concomitant with choice difficulty. Behaviorally, we found that the average individual’s subjective effort cost exhibited increasing marginal costs as effort increased. Neurally, we found that activity in vmPFC was related to the subjective valuation of prospective effort, while ACC activity was best described by choice difficulty. These results suggest that vmPFC encodes the subjective costs that underlie choices involving physical effort, and ACC activity is related to the cognitive control required at the time of choice.

Our findings are consistent with the idea that vmPFC encodes a general subjective value signal, which subserves effort decisions, similar to the value signals that have been previously reported for a variety of appetitive and aversive stimuli (Bartra et al. 2013; Clithero and Rangel 2014; O’Doherty 2014). Notably, vmPFC has also been implicated as a potential hub for making comparisons between similar and different types of value signals (Chib et al. 2009; Lebreton et al. 2009; Levy and Glimcher 2011), which is consistent with our finding of relative effort value signals in vmPFC. Together these findings add to the understanding of vmPFC function by showing that this brain region not only encodes appetitive and aversive stimulus values of economic goods but also the subjective value of prospective physical effort.

The effort value signals we observed in vmPFC also align with the concept that this region is involved in a “goods space” representation of value, in which abstract value representations are encoded separately from the necessary motor plans for execution (Padoa-Schioppa 2011). The goods-based model proposes that the values of stimuli are compared directly to make a choice, and only after a stimulus is chosen are the necessary motor processes identified and executed. Thus, the goods-based model suggests a sequential choice process in which action selection (and engagement of motor processes) are temporally separated from the process of choice. We failed to find brain activity in motor and premotor regions at the time of choice, which aligns with the neural separation of decision value signals and engagement of action plans and motor processes. However, we are cognizant that a negative finding in motor regions does not provide definitive evidence against the presence of action values.

Previous studies of effort cost have focused on the trade-offs between prospective effort and reward, similar to the natural choices we make in everyday life (Croxson et al. 2009; Kurniawan et al. 2010, 2013; Prévost et al. 2010; Skvortsova et al. 2014; Bonnelle et al. 2016; Klein-Flügge et al. 2016; Chong et al. 2017) and have suggested that in these contexts ACC encodes effort cost. However, these studies were not designed to isolate effort valuation from reward and instead focused on the integration of both of these utilities to compute a decision. In our paradigm, however, we took a reductionist scientific approach, isolating effort valuation in order to provide a computational description of how subjective valuation of effort is encoded in the brain. While our choice paradigm is not as naturalistic as the effort/reward trade-offs made in daily life, such an approach is valuable because it affords us a deeper understanding of an individual component (in this case effort valuation) that shapes more complex decisions. Our paradigm revealed that activity in vmPFC was better represented by subjective, rather than objective, valuation of prospective effort. This is consistent with previous research in an intertemporal choice setting that found this region better encodes subjective monetary values compared with objective monetary values (Kable and Glimcher 2007).

Notably, we found that ACC activity in our task was better described by choice difficulty than the subjective valuation of effort. These findings are consistent with previous studies of neuroeconomic choice that experimentally separated prospective value and choice difficulty and found the former to be encoded in ACC. There is an ongoing debate regarding the role of ACC in decision-making and whether it encodes decision values or variables related to cognitive control (e.g., choice difficulty) (Ebitz and Hayden 2016; Kolling, Behrens et al. 2016; Kolling, Wittmann et al. 2016; Shenhav et al. 2016). With this debate in mind, it has been proposed that when studying valuation it is important to design studies that are capable of separating the two.

A few recent studies of effort/reward trade-offs have explored brain activity related to choice difficulty and effort valuation. These studies have reported mixed results. While all of these works reported effort valuation signals in ACC, the studies that performed control analyses to explore choice difficulty failed to find significant signals in ACC (Klein-Flügge et al. 2016; Chong et al. 2017), and another study reported both effort valuation and choice difficulty signals in ACC (Bonnelle et al. 2016). Notably, the studies that failed to find choice difficulty signals in ACC used experimental paradigms that did not span the full space of prospective effort to elicit a full range of choice preferences, making it challenging to dissociate signals related to valuation and choice difficulty. The study that reported both effort value signals and choice-difficulty-like (i.e., cost–benefit) signals in ACC utilized a paradigm in which choices about effort immediately preceded exertion, so it is difficult to know if these ACC signals were related to motor anticipation/preparation (Nguyen et al. 2014) or effort valuation per se. Overall, these previous studies of effort-based decision-making were not specifically designed to study the variables of prospective effort valuation and choice difficulty while also eliminating the requirement of effort/reward trade-offs. This makes it difficult to determine if the neural effort value signals observed in these studies are truly related to effort valuation alone, effort values translated into a monetary scale, or multiplexed effort/reward signals. Notably, a recent study which carefully controlled for these factors, in the context of decisions about prospective cognitive effort, has suggested that vmPFC encodes subjective value of effort while ACC encodes choice difficulty (Westbrook et al. 2018).

Our paradigm eliminated the need for effort/reward trade-offs, was designed to span the full space of effort valuation and choice preference, and fully dissociated neural processing related to prospective effort valuation from exertion. In doing so, we found that ACC activity was better described by choice difficulty than effort valuation. Our result of effort-based choice difficulty signals in ACC, taken together with previous studies of foraging (Shenhav et al. 2014, 2016) are consistent with ACC’s role in cognitive control during decision-making. In particular, our findings align with conflict monitoring theories that suggest ACC tracks the level of indifference in decision-making tasks because higher indifference requires increased cognitive control (Botvinick et al. 2001; Botvinick 2007). It is important to mention that it is also possible that such conflict signals could be a by-product of ACC comparing the values of the options presented (Hare et al. 2011).

It has also been proposed that the ACC activity found in previous studies could be indicative of a multiplexing node that combines action/reward values (Hayden and Platt 2010; Shenhav et al. 2013; Klein-Flügge et al. 2016) and serves as a gateway that informs the motor system to act for reward (Cai and Padoa-Schioppa 2012). Thus, it is possible that our lack of significant subjective effort value signals in ACC could also be due to the fact that our study was designed to isolate neural representations of subjective effort valuation that were “independent” of (and not multiplexed with) reward.

Our results suggest that in the context of effort-based decision-making, in which rewards are not present and choice difficulty is controlled, the ACC better represents choice difficulty than effort valuation. While this finding is in contrast to previous studies of effort/reward trade-offs that have implicated ACC in this process, it is important to stress that our study attempted to present effort-based decisions in the absence of appetitive rewards and thus was quite different from those previous investigations. While our paradigm provides insights into the fundamental valuations of effort, controlling for a number of factors, it is important to recognize that it does not allow us to make definitive claims about how neural value signals for effort and reward are integrated to subserve effort-based choice. Future studies will be needed to determine, if when controlling for choice difficulty in more naturalistic paradigms involving effort reward trade-offs that span the full space of exertion and reward, ACC activity is still best described by difficulty and vmPFC by effort valuation. Notably, in the ongoing debate about ACC function during economic choice, it has been suggested that subregions within ACC might simultaneously encode signals about valuation and choice difficulty (Kolling, Behrens et al. 2016; Kolling, Wittmann et al. 2016), and in this vein it is possible that ACC could encode both effort costs and choice difficulty. In fact, this is also a distinct possibility in our experimental paradigm given that we found a trend-level relationship between ACC activity and RV_sure (although this relationship did not reach significance).

In this study, we focused on characterizing the subjective valuation of physical effort in the form of grip force. However, an individual’s subjective effort costs could vary across types of effort (i.e., walking, arm movements, or even cognitive effort) in a similar fashion to how individuals exhibit different subjective values for different types of goods (Chib et al. 2009; Levy and Glimcher 2011). Moreover, just as the subjective value of rewards can be modulated by the state of an individual, subjective costs of effort could also be influenced by state. For example, individuals having undergone physical or cognitive fatigue or training might exhibit modified representations of subjective effort cost. Furthermore, it is possible that the subjectivity of different types of effort may exhibit similar trait-like consistency over time, as has been reported in studies of subjective valuation of money (Ohmura et al. 2006; Kable and Glimcher 2007; Ballard and Knutson 2009).

Characterization of subjective effort costs will provide an understanding of why some people find certain tasks to be very effortful while others complete them with ease. Such knowledge could be used to design incentive mechanisms that account for perceptions of effortfulness to maximize employees’ performance. Insights into these preferences may aid in the development of more efficacious individual-specific behavioral mechanisms that enhance motivational output and effort exertion in a variety of everyday tasks.

Funding

This work was supported by the Eunice Kennedy Shriver National Institute Of Child Health & Human Development of the National Institutes of Health under Award Number K12HD073945 to V.S.C. J.G. was supported by the National Defense Science and Engineering Graduate Fellowship. The authors declare no competing financial interests.

Notes

The authors thank C. Frydman for his help with the design and analysis.