Effort discounting in human nucleus accumbens

Abstract

A great deal of behavioral and economic research suggests that the value attached to a reward stands in inverse relation to the amount of effort required to obtain it, a principle known as effort discounting. In the current report, we present the first direct evidence for a neural analogue of effort discounting. Functional magnetic resonance imaging was used to measure neural responses to monetary rewards in the human nucleus accumbens, a structure previously demonstrated to encode reference-dependent reward information. The magnitude of accumbens activation was found to vary with reward outcome, but also with the degree of mental effort demanded to obtain individual rewards. For a fixed level of reward, accumbens was less strongly activated following a high demand for effort than following a lower demand. The magnitude of this effect was noted to correlate with preceding activation in the dorsal anterior cingulate cortex, a region that has been proposed to monitor information-processing demands and to mediate in the subjective experience of effort.

At the level of everyday commonsense knowledge, there is a close relationship between reward and effort. This is evident, for example, when a potential payoff is judged not to be worth the work it would require, or when a level of reward for some effort is considered to be unfair. The same direct connection between reward and effort is also found in formal theories, including behavioral and economic accounts of decision making (see Kivetz, 2003; Walton, Kennerley, Bannerman, Phillips, & Rushworth, 2006), social theories of equity (Walster, Walster, & Berscheid, 1978), and legal theories of distributive justice (Locke, 1987). In these contexts and others, a common proposition is that effort carries a negative value or cost, sometimes referred to as the disutility of effort, and that this cost provides a reference against which earned rewards are evaluated (Figure 1). Under this basic principle, referred to in some contexts as effort discounting, a reward carries a higher net value if it is easily obtained than if it is obtained only through great effort (Kivetz, 2003; Phillips, Walton, & Jhou, 2007; Rudebeck, Walton, Smyth, Bannerman, & Rushworth, 2006). In effect, effort sets in place a reference point against which rewards are measured.

Figure 1.

Two diagrammatic representations of effort discounting, one from animal behavior research and one from behavioral economics. Top: Based on choice behavior in animals, Phillips et al. (2007) proposed that the subjective value (net utility) of a fixed reward varies inversely with the effort required to obtain it (response cost). They review work suggesting that the strength of this discounting effect varies with concentrations of dopamine (DA) within the NAcc (Redrawn from Figure 2 of Phillips et al., 2007). Bottom: According to prospect theory (Tversky & Kahneman, 1992), the relationship between a reward’s magnitude and its subjective value is characterized by a curvilinear value function. Kivetz (2003) proposed that this function shifts to the right as the effort demanded to obtain the reward increases. The figure shows two value curves, one relevant to rewards requiring a fixed amount of effort (right), the other to rewards requiring no effort (left; horizontal axis: reward magnitude including both wins and losses, with zero at the point labeled r=0; vertical axis: subjective value, with the horizontal axis crossing at zero. From Figure 1 of Kivetz, 2003).

Effort discounting, and the close relationship between reward and effort that underlies it, clearly represent more than mere cultural convention. Rodents, birds and non-human primates have been shown to weigh effort against reward in decision making (Phillips et al., 2007; Salamone, Cousins, & Bucher, 1994; Stevens, Rosati, Ross, & Hauser, 2005; Tsunematsu, 2000; Walton, Bannerman, Alterescu, & Rushworth, 2003; Walton et al., 2006; some contrary conclusions from the same literature are discussed below), and capuchin monkeys have been shown to reject rewards smaller than those received by conspecifics for an equal expenditure of effort (Brosnan & De Waal, 2003). Psychopharmacologic interventions and lesions to specific brain structures have been observed to alter the relative weighting of effort and reward information in decision making (Denk et al., 2005; Floresco & Ghods-Sharifi, 2006; Salamone, Correa, Mingote, & Weber, 2003; Salamone et al., 1994; Walton et al., 2003). Furthermore, effort discounting relates closely to another form of discounting — delay discounting — for which specific neural substrates have been identified (see, e.g., Roesch, Taylor, & Schoenbaum, 2006; Rudebeck et al., 2006). Given such findings, it seems plausible that the tight relationship between reward and effort that holds at the behavioral level may reflect the operation of basic neural mechanisms.

In the present experiment, we used functional magnetic resonance imaging (fMRI) to investigate the integration of reward and effort information in the human brain. Our specific objective was to test for a neural correlate of effort discounting. The experiment focused on the nucleus accumbens (NAcc), a basal ganglia structure that has been found in numerous studies to respond to reward outcomes, often in a reference-dependent fashion (Breiter, Aharon, Kahneman, Dale, & Shizgal, 2001; Delgado, Nystrom, Fissell, Noll, & Fiez, 2000; Elliott, Friston, & Dolan, 2000), and one that has also been heavily implicated in effort-based decision making (Salamone, Correa, Farrar, & Mingote, 2007; Salamone et al., 2003; Salamone et al., 1994; Walton et al., 2006). In alignment with a recent study of delay discounting (Roesch et al., 2006), we looked for discounting effects at the time of reward receipt, following effort expenditure. Based on theories of effort discounting (e.g., Kivetz, 2003; Phillips et al., 2007) we hypothesized that the NAcc response to reward would vary inversely with the level of effort demanded prior to reward delivery.

Although effort discounting has been considered to result from both physical and mental effort, our focus in the present experiment was on mental effort. One advantage of this choice is that relatively solid information is available concerning the neural response to both cognitive demand and the associated subjective sense of mental effort. In particular, these have been proposed to engage the dorsal anterior cingulate cortex (ACC; Botvinick, Braver, Barch, Carter, & Cohen, 2001; Botvinick, Cohen, & Carter, 2004; Naccache et al., 2005). Studying the effects of mental effort on reward processing thus presented the opportunity to compare neural encodings of reward (within the NAcc) to encodings of effort itself (within the ACC).

Methods

Participants

The experiment was performed at the University of Pennsylvania. A total of 45 participants were recruited from the university’s student and staff population. All denied any history of neurological or psychiatric conditions. Participants were randomly sorted into an experimental group (30 total, 22 females) and a control group (15 total, 8 females).¹ Of the 30 experimental participants, 7 (all females) were ultimately excluded from analysis, three due to scanner malfunction and four due to excessive movement, based on criteria described below. Of the 15 control subjects, two were excluded due to excessive movement. All participants included in the final analysis were right-handed and ranged in age from 19 to 33. They provided informed consent and were paid for their participation.

Behavioral task

The experimental protocol was approved by the University of Pennsylvania School of Medicine institutional review board. Participants in both experimental and control groups performed in a task-switching paradigm involving probabilistic reward (Figure 2). In each task block, participants viewed a series of ten individually presented numerals, ranging from 1 to 9 but excluding 5. All visual stimuli were displayed on a back-projection screen placed at the head of the scanner bore, which was viewed through a mirror mounted on the head coil. The numerals were presented on a black background in 72-point Arial font. Subjects responded to each numeral by pressing one of two buttons on a fiber-optic response pad. The correct response depended on the color in which the numeral was presented. If yellow, subjects were to perform a parity judgment, depressing the button beneath their right index finger to indicate even and a button beneath their right middle finger to indicate odd. If the target numeral appeared in blue, subjects used the same two buttons to perform a magnitude judgment (index finger: less than five, middle finger: greater than five).

Figure 2.

The sequence of events within low demand (left) and high demand (right) blocks. Each block included ten numerals, and could end with either a $ or an X.

The experiment involved two types of task block. In low demand blocks, all digits were of the same color. In high demand blocks, the color alternated across successive digits, requiring the subject to make effortful and inefficient switches between tasks (see Monsell, 2003). In a parallel behavioral study, reported elsewhere (Botvinick, 2007), subjects given a free choice between these two task conditions showed a consistent bias against the high-demand option, consistent with the idea that this carried relatively high disutility.

Each block was preceded by a brief visual cue that indicated whether the upcoming block would be of the high or low demand type. A solid disk (yellow or blue) indicated a low demand block, and additionally indicated the relevant task. A vertically split disk, half yellow and half blue, indicated a high demand block, and provided no information about the color of the first digit. Within each scan and over the course of the experiment as a whole, the two block types and the two classification tasks occurred with equal frequency, following a different randomized sequence for each subject.

Immediately following the last digit in each block, a message appeared, reading “Deciding your pay….” After a variable interval, this was replaced with a large white dollar sign, indicating that the participant had earned one dollar for the just-completed block, or a large white X, indicating that nothing had been earned for the block.

The specifics of stimulus timing were as follows. Disk cue duration: 2000 ms. Digit duration: 1500 ms, with a 500 ms interstimulus interval. Message duration ranged arbitrarily from 2000 to 8000 ms, in 2000 ms steps. Reward cue duration: 2000 ms. Interblock interval ranged arbitrarily from 2000 ms to 8000 ms, in 2000 ms steps.

At the beginning of each session, the experimenter read the instructions aloud to the subject. Participants were then given five minutes of practice on the task-switching paradigm, without reward cues. During the fMRI experiment, participants performed the task (with rewards) throughout four scanning periods, each approximately ten minutes in duration (see Image Acquisition). Each scanning period contained 18 task blocks. Within each period, the sequence of high and low demand blocks, and of $ vs. X cues, was randomly generated, subject to the constraint that each scan could contain no more than five and no fewer than three occurrences of any specific combination of block type and cue type. This resulted in roughly equal numbers of each block/cue combination over the course of the experiment. Note that it was important to prevent significant imbalances in reward rate between high and low demand blocks, so as to avoid corresponding differences in reward prediction. At the same time, to assure that subjects would take interest in the reward cues, it was necessary to create the impression that the range of possible cumulative rewards was fairly wide. With this in mind, participants were told prior to participation that they would earn between $10 and $50, and as part of the instructions they were told that the amount of pay would depend directly on the number of $ cues that occurred during their session. All subjects received $40 for their participation (slightly greater than the number of $ cues, which ranged from 32 to 39).²

Prior to beginning the fMRI experiment, participants were informed that the rate of ‘pay’ would be equal for high and low demand blocks. In addition, it was made clear that reward would not be contingent upon speed or accuracy during task performance. Nevertheless, participants were encouraged to respond as quickly as possible without sacrificing accuracy.

The decoupling of reward probability from reaction time, error rate and block type was a critical aspect of the experimental design, and merits further comment. Note, first, that rewards were not intended as incentives to effort. Effort was manipulated entirely through externally imposed task demands, rather than through differential incentives. Our interest was in how external demands for effort affect reward processing, rather than in how rewards affect the voluntary mobilization of effort on a fixed task. In order to prevent confusion on this point, we shall henceforth tend to refer to the difference between block types in our experiment as relating to “demand” rather than “effort.”

Second, note that if reward had been made dependent on reaction times or error rates, this would have introduced differences in reward rates between high and low demand blocks, because reaction times and error rates could be expected to differ between the two. This, in turn, would likely have led to different expectancies for reward between the two block types. Specifically, subjects would have grounds to predict a $ outcome on low demand blocks more strongly than on high demand blocks, simply based on observed frequencies. This is important, because NAcc reward responses have been shown to vary depending on reward expectancy (see, e.g. Breiter et al., 2001). Thus, making reward probability dependent on reaction times or error rates would have introduced a serious confound.

It was also important to inform subjects that reward probabilities would be equivalent between block types in order to prevent the possible, culturally-based assumption that the experimenters would reward high demand blocks with more frequent $ outcomes than low demand blocks. The idea of effort discounting suggests, loosely speaking, that rewards are valued in reference to what is felt to be deserved, not only what is predicted. By explicitly informing subjects of the equivalent reward probabilities between block types, we sought to neutralize the latter factor (prediction) in order to focus on the former (entitlement).

Note that it was possible, in principle, for subjects to conceive of the reward events as completely independent of the temporally associated task performance blocks. However, pilot work and informal subject interviews suggested that this was prevented by framing rewards as ‘pay.’ In order to understand why this framing may have been effective, consider that pay-for-work in everyday life is rarely immediately contingent on details of work performance. Thus, a factory worker’s hourly pay is not typically immediately dependent on the worker’s hourly output. In general, in our culture (indeed, including the setting of the typical psychology experiment), one is usually paid a pre-established and fixed amount for performing work within certain, usually implicit, performance bounds. This was precisely the situation in our experiment. In any event, it was considered that any effort discounting effect observed in the fMRI data would provide supportive, if not indisputable, evidence that subjects had represented reward events as related to task-block completion.

The control version of the task was designed to verify that any effort discounting effect observed in the experimental group was specific to reward processing, and not due to nonspecific or incorrectly modeled effects of demand on NAcc activation. The control task was identical to the experimental task, with two exceptions. First, the message at the end of each block read “Calculating, please wait….” Second, the message was followed by an S or a K, rather than a $ or X. Control participants were told that the message and these letters were for the information of the experimenter, and were irrelevant to the participant’s task. However, the instructions were nonetheless to pay attention during the letter cues, “to make sure you don’t miss the beginning of the next block.” The instructions prior to the experiment did not mention any block-by-block rewards for task performance. Subjects were told they would earn $20 for their participation, and they received this amount at the end of the experiment.³

Image acquisition

Functional magnetic resonance images were collected using a Siemens Trio scanner operating at 3 Telsa, with a Siemens 8-channel head coil. Each scanning session began with the acquisition of a high resolution, T1-weighted axial anatomical image. The anatomical scan was followed by four scans of axial gradient-echo echoplanar images (EPI) with the following specifications: TR = 2000 ms, TE = 30 ms, α = 90°, matrix = 64 × 64, FOV = 24 cm, 3×3×3 mm voxels. Thirty-three contiguous slices were collected during each TR. Four functional scans were completed, each containing 306 TRs.

Image analysis

Offline data processing was performed using the VoxBo software package (www.voxbo.org). After image reconstruction, anatomical images were corrected for field inhomogeneities and transformed to standardized space, defined by a Montreal Neurological Institute (MNI) template. The four-dimensional functional time-series data were interpolated to correct for slice acquisition timing and spatially realigned to the first TR of the first scan of the study, to correct for subject movement, spatially normalized, and spatially smoothed using a 4 mm fixed-width at half-maximum kernel. Within each scan, signal at each voxel was mean normalized and linearly detrended. Data for participants showing greater than 2 mm movement in any plane during a single functional scan were excluded from further analysis.

Subsequent analysis focused on five a priori regions of interest (ROI). A bilateral NAcc ROI (mean voxel count 35) was defined according to the guidelines proposed by the Massachusetts General Hospital Center for Morphometric Analysis (http://www.cma.mgh.harvard.edu/). Four further ROIs were defined, for comparison with NAcc (see Results for further comments on selection of these ROIs). A medial prefrontal cortex ROI (44 voxels) was estimated from Table 1 and Figure 1 of Knutson et al. (2001), an area reported in that study to show sensitivity to cues indicating reward outcomes.

Table 1.

Summary of results from ROI analyses. Plus signs indicate that the relevant effects were observed in the experimental group (p < 0.05). The parenthesized plus sign indicates that the relevant effect did not reach significance in the experimental group, but did do so when the experimental group was compared with the control group.

ROI	Effect
	Reward Cue	Block Type	Interaction
Nucleus Accumbens	+	+	−
Orbitofrontal cortex	+	−	−
Medial prefrontal cortex	+	−	−
Amygdala	(+)	−	−
Dorsal Anterior Cingulate	−	+	−

The resulting ROI was centered at Talairach coordinates (3, 40, −12). (Talairach coordinates were mapped to MNI space using the tal2mni function developed at Cambridge University, see http://imaging.mrc.cbu.cam.ac.uk/imaging/MniTalairach). A bilateral ACC ROI (206 voxels) was based on a meta-analysis of studies implicating the ACC in cognitively demanding tasks (Ridderinkhof, Ullsperger, Crone, & Nieuwenhuis, 2004). The region chosen was based on Figure 1B of that article, and was shaped so as to encompass the region of highest density within the cluster of ACC locations depicted in that figure (centered approximately at Talairach coordinates 0, 14, 41). The resulting ROI corresponded closely to a region identified as involved in reward evaluation in a study by Bush et al. (2002). Orbitofrontal and amygdala ROIs were drawn based on inspection of anatomical images. Coverage of OFC was adequate in 22 of 23 experimental subjects. In initial analyses, separate ROIs were drawn for medial and lateral divisions of OFC (mean voxel counts 190 and 821, respectively), following previous work suggesting differential function across this divide (e.g., O’Doherty, Kringelbach, Rolls, Hornak, & Andrews, 2001). However, data from these sub-regions was pooled after no significant difference was found in their response to reward cues ($ vs. X). A left DLPFC ROI (82 voxels) centered at Talairach coordinates (−48, 21, 21) was drawn from Luks et al. (2002), which reported activation within this region in association with task-switching, the source of cognitive demand in our study.

Within each ROI, spatially averaged signal time-courses were passed through a notch filter, removing frequencies above 0.25 Hz and below 0.0032 Hz. The resulting time-courses for each ROI were then analyzed using the general linear model (GLM) as implemented in VoxBo. The model included covariates for 10 task events: solid disk cue, split disk cue, digit within high demand block, digit within low demand block, message following high demand block, message following low demand block, $ following high demand block, $ following low demand block, X following high demand block, and X following low demand block. Each of these covariates was convolved with a canonical hemodynamic response function. The intertrial interval provided a baseline. The model also included covariates for intercept, scan effects, and any individual mean signal spikes greater than 3.5 deviations from the experiment mean.

Parameter estimates (beta values) were extracted for each ROI for each event-type. The resulting values were entered into further analyses, as enumerated in the Results section. ROI analyses of reward-cue responses assumed the form of a repeated measures analyses of variance (ANOVA), with factors for block type (high vs. low demand) and reward cue ($ vs. X). Analyses comparing reward-cue activity between experimental and control groups took the form of a repeated measures analyses of covariance (ANCOVA), with factors for block and cue type ($ and S vs. X and K) and for participant group. These analyses also included, as a covariate, the difference between the parameter estimates for ROI activation during high-demand and low-demand block performance.⁴ Analyses comparing reward-cue responses between regions of interest involved repeated measures analyses of variance with factors for block type, cue type and ROI.

In order to further establish regional specificity of findings from earlier analysis, a whole-brain exploratory analysis was also conducted. Here, data were preprocessed and smoothed as described above. To each voxel’s timecourse we applied the same general linear model that was applied to average time-courses in the foregoing ROI analyses. Each subject’s four parameter estimates for reward cues were then taken to group-level analyses. Single-subject general linear models were estimated using VoxBo software; group-level ANOVAs were carried out using AFNI (Cox, 1996). Specific contrasts, and associated significance thresholds are described under Results.

As discussed further under Results, a third set of GLM analyses was conducted in order to examine correlations between ACC activation during numeral classification, within individual blocks, and the NAcc response to the subsequent reward cues. The model here included individual covariates for each task block and for each reward cue, as well as one covariate for all disk cues and one covariate for all message events. Parameter estimates for the reward cue response within the NAcc ROI were entered into a linear regression with regressors for cue type, block type and ACC parameter estimate, as well as for the NAcc parameter estimate for numeral-task performance in the block preceding each reward cue. The latter covariate was included in order to partial out gradual fluctuations in NAcc activation.

Results

Validation of Effort Manipulation

Both behavioral and fMRI data were consistent with the expectation that task blocks involving continual task-switching would be more difficult than blocks requiring no switching. Manual responses during high demand blocks were slower and less accurate than those during low demand blocks (mean reaction time 853 ms vs. 645 ms, t(22) = −15.6, p < 0.0001; mean percent correct 93.7% vs. 96.6%, t(22) = 3.31, p = 0.003). This effect held for both parity and magnitude judgment tasks (parity: low demand, 664 ms, 96.5% correct; high demand 866 ms, 92.7%; magnitude: low demand 614 ms, 96.7%; high demand 837 ms, 94.7%). In a questionnaire completed at the end of the experiment, participants rated high demand blocks more effortful (mean 7.11 vs. 3.78 on a scale from 1 to 10; Wilcoxon signed-ranks test, Z = −4.15, p < 0.0001). The questionnaire also asked participants to indicate what they believed would have been “fair pay” for performance of individual high demand and low demand blocks This question was included in the questionnaire only for the last 9 subjects. Participants consistently assigned a higher dollar amount to high demand blocks (mean $1.89 vs. 89¢, t(8) = 6.93, p = 0.0001), consistent with the idea that the effort required by high demand blocks was associated with significant disutility, and also consistent with the choice data obtained in the parallel behavioral study (see Methods).

A further indication that the demand manipulation was successful was that task performance induced greater activation in high demand blocks than in low demand blocks within dorsal anterior cingulate cortex (ACC; t(22) = 7.00, p < 0.0001), a region widely considered to monitor task difficulty and demand for cognitive control (Botvinick, Nystrom, Fissell, Carter, & Cohen, 1999; Botvinick et al., 2001; Botvinick et al., 2004; Davis et al., 2005; Kerns et al., 2004; Ridderinkhof et al., 2004) and also proposed to mediate in the subjective experience of mental effort (Naccache et al., 2005).

In order for the control group to serve its intended role, it was important to confirm that subjects in that group exerted themselves no less than subjects in the experimental group. Consistent with this, reaction times and error rates revealed no significant differences between the control and experimental groups. Indeed, both reaction times and error rates were numerically, if not statistically, lower in the control group (high-demand, 783 ms, 95.5% correct; low-demand, 591 ms, 98.2% correct; main effect of group, F(1,33) = 3.59, p = 0.07 for reaction time; F(1,33) = 0.83, p = 0.37 for error rate).⁵ Ironically, this pattern raises the unanticipated possibility that subjects in the control group may have exerted slightly greater effort overall than subjects in the experimental group. In light of this, it is critical to note that no interaction was found between subject group and block type, consistent with the conclusion that the difference in demand between the two block types was comparable between groups (reaction time, F(1,33) = 0.83, p = 0.37; error rate, F(1,33) = 0.03, p = 0.87). Also consistent with this conclusion were the findings that, within the control group, ACC activation during task blocks was greater for high demand than low demand blocks (t(12) = 4.28, p = 0.001) and that the size of this effect did not differ significantly from the experimental group (t(34) = −0.88, p = 0.39).

Effort Discounting in NAcc

In order to analyze responses to reward cues, activation values within NAcc were entered into a two-way repeated measures ANOVA, with factors for cue ($ vs. X) and preceding block type (high vs. low demand; see Methods). Consistent with previous studies, the NAcc activated more strongly in response to the $ cue than the X (main effect of cue, F(1,22) = 10.97, p = 0.003).⁶ More critically, NAcc activation also showed a main effect of block type, according to which activation in response to reward cues was higher following low demand blocks than high demand blocks (F(1,22) = 10.32, p = 0.004; Figure 3A). This difference was evident both for $ cues (t(22) = 2.22, p = 0.037) and for X cues (t(22) = 3.09, p = 0.005). Indeed, in the high-demand condition, the X cue was associated with a relative decrease in NAcc activation, consistent with the idea that, under sufficient effort demands, a missed reward can in fact register as a loss (see Figure 1, bottom, and Kivetz, 2003). The context-dependence of NAcc responses to cues marking zero reward also fits with previous studies in which NAcc responses to such cues were affected by reward expectancies (Breiter et al., 2001; Knutson, Taylor, Kaufman, Petersen, & Glover, 2005).

Figure 3.

(A) Top left: Coronal section showing the location of the NAcc ROI. NAcc: Mean parameter estimates for NAcc activation in response to $ and X cues occurring at the completion of high demand and low demand blocks, after subtraction of corresponding parameter estimates from control condition. Bars indicate standard error. OFC: Orbitofrontal cortex. MFC: Medial prefrontal cortex. AMG: Amygdala. ACC: Anterior cingulate cortex. (B) Top: Coronal section showing the location of the ACC (green) and DLPFC (blue) regions of interest. Bottom: Regression coefficients for each participant in the experimental group, relating reward-cue responses in NAcc to task-related activity in ACC and DLPFC. Position along the horizontal axis reflects the relation between ACC activation during task performance and the NAcc response to the subsequent reward cue. Position along the vertical axis reflects the relation between DLPFC activation during task performance and the NAcc response to the subsequent reward cue. A more consistent relationship was observed between ACC and NAcc than between DLPFC and NAcc. (C) Mean NAcc responses to the visual cues occurring at the beginning of high and low demand blocks, for experimental and control groups. Bars indicate standard error.

Although the difference between the responses to $ and X cues was larger on average following high demand blocks than low demand blocks, the factors for reward cue and block type did not show a significant interaction in the ANOVA (F(1,22) = 1.54, p = 0.24). Note that such an interaction would follow from some theories of effort discounting, but not others. Consider, for example, the two accounts diagrammed in Figure 1. The account illustrated in the upper panel of the figure does not predict an interaction, since here net utility is a linear combination of gross reward and effort cost. In contrast, the theory depicted in the lower panel of Figure 1 does predict an interaction.⁷ This is due to the shape of the utility function shown in the figure, which builds in an assumption of loss aversion (by assuming a stronger curvature above the x-axis than below). Of course, both of these predictions depend on the assumption that there is a linear relationship between net utility and NAcc BOLD, and in this respect the absence of an interaction in our data must be interpreted with caution. The material point is that the absence of a significant interaction in our data does not contradict any strong prediction arising from the idea of effort discounting.

The absence of a significant interaction between demand and reward cue in NAcc does, however, raise one important interpretive issue. Specifically, it leaves open the possibility that the apparent discounting effect in NAcc might instead reflect baseline differences between the high and low demand conditions, i.e., a simple carryover of (or rebound from) activation in NAcc arising during block performance. Fortunately this interpretation is ruled out by data from the control group, as discussed next.

Comparison with control group

Results from the control group indicated that the effect obtained in the experimental group was specific to reward processing, and not due to nonspecific or incorrectly modeled effects of effort on NAcc activation. A two-way ANOVA on NAcc activation in this control condition (see Methods) revealed no significant main effect of either cue type (S vs. K; F(1,12) = 2.72, p = 0.13) or block type (F(1,12) < 0.001, p = 0.99). Pooling across cue types, mean and standard error for NAcc activity (percent change) were 0.12 and 0.09 in low demand blocks, and 0.012 and 0.10 in high demand blocks.

In order to directly compare the profile of NAcc activation in the experimental and control groups, a mixed-effects ANCOVA was conducted including within-subject factors for cue and block type and a between-subjects factor for participant group (experimental vs. control; see Methods). This revealed a significant interaction between block type and participant group (F(1,34) = 16.09, p = 0.0003), supporting the conclusion that the impact of demand on the response to the ensuing reward cue differed between experimental and control groups.

Note that this difference between groups was not explained by any difference in NAcc activity leading up to reward-delivery events. In both groups, NAcc activity during block performance was slightly higher during low demand than high demand blocks. This effect was at trend level in the experimental group, t(22) = 1.87, p = 0.074, and statistically significant in the control group, t(22) = 2.77, p = 0.011. However, the size of this difference between block types did not significantly differ between groups, t(34) = 1.06, p = 0.30. Moreover, the magnitude of the difference, for individual subjects, was included as a covariate in the ANCOVA comparing the experimental and control groups (see Methods). Thus, the difference in the discounting effect between groups is unlikely to reflect a carry-over from earlier task-induced differences in NAcc activity.

Comparison with other Regions

In order to investigate the anatomical specificity of the activation pattern observed in the NAcc, we evaluated activation in four other regions of interest, two associated in previous research with processing of reward but not demands for effort, and two associated with both reward and effort processing.

The orbitofrontal cortex (OFC) has been extensively implicated in the representation of reward information (e.g., Rolls, 2004; Schultz, Tremblay, & Hollerman, 2000). However, Rudebeck et al. (2006) provided evidence that OFC does not play a critical role in the representation of effort costs. Consistent with this, we found that OFC activity was significantly affected by reward information (main effect of reward cue, F(1,21) = 8.85, p = 0.007), but not by the level of effort demanded (main effect of block type, F(1,21) = 2.12, p = 0.160; Figure 3A).⁸ The main effect of demand observed in NAcc differed significantly from the pattern in OFC (F(1,21) = 5.29, p = 0.032).

An analogous pattern was observed in medial prefrontal cortex (MFC), within a rostral region previously reported to respond to reward outcomes (Knutson et al., 2001; Rogers et al., 2004), but not previously associated with the processing of effort demands (Figure 3A). Like OFC, this region showed a main effect of reward cue (F(1,22) = 4.84, p = 0.039), but not of block type (F(1,22) = 0.36, p = 0.55), The main effect of demand observed in NAcc differed significantly from the pattern in MFC (F(1,22) = 14.37, p = 0.001).

The basolateral nucleus of the amygdala has been implicated in both reward (see Baxter & Murray, 2002) and effort (Floresco & Ghods-Sharifi, 2007) processing. In our experiment, although the main effect of reward cue did not reach significance in the experimental group (F(1,22) = 2.45, p = 0.13), the amygdala response to $ vs. X did differ significantly from control (F(1,33) = 4.68, p = 0.038). However, the amygdala showed no significant effect of block type (F(1,22) = 0.76, p = 0.39), and the pattern here differed significantly from that seen in NAcc (F(1,22) = 5.75, p = 0.025).

A different profile was observed in the dorsal ACC, another region associated with both reward (e.g., Bush et al., 2002; Shidara & Richmond, 2002) and effort (e.g., Botvinick, 2007; Botvinick et al., 2004; Walton et al., 2003) processing. The ACC showed a main effect of block type (F(1,22) = 12.15, p = 0.002), but no significant main effect of reward cue (F(1,22) = 1.19, p = 0.29).⁹ The response of the NAcc to reward cues differed significantly from that of the ACC (F(1,33) = 9.80, p = 0.005). The ACC did show a trend toward interaction between reward and block type (F(1,22) = 3.01, p = 0.10). However, the interaction pattern did not significantly differ from the one observed in controls (F(1,33) = 0.04, p = 0.83).

The overall pattern of findings is summarized in Table 1. In short, three regions (OFC, MFC and amygdala) displayed some sensitivity to reward, and one (ACC) appeared to encode effort costs. However, among the regions studied, only NAcc showed both effects. The effort-discounting effect observed in the NAcc thus appears to be regionally specific.

To further test the specificity of the pattern obtained in NAcc, we conducted an exploratory analysis as detailed under Methods. The intention of this analysis was to localize joint main effects of cue and block-type, examining whether these appeared in regions outside the NAcc. However, consistent with the assumptions about statistical power that prompted an initial ROI-based analysis, no region (including the NAcc) passed conventional Bonferroni-corrected significance thresholds either for this conjunction of main effects or for the interaction effect between cue and block-type. As a follow-up measure, we identified the lowest uncorrected F-value threshold at which at least one voxel in both left and right NAcc survived for both main effects. This threshold turned out to be F(1,22) = 9.67, corresponding to an uncorrected p value for each main effect of 0.005. We then surveyed the rest of the brain for voxels elsewhere that survived this same threshold. A total of 118 such voxels were found, 92 in the occipital cortex (right calcarine gyrus, 13; middle gyrus, 26 right, 32 left; inferior gyrus, 15 right, 4 left; left superior gyrus, 2), 19 in the parietal cortex (superior lobule, 4 left, 8 right; left inferior lobule, 2; right supramarginal gyrus, 1; right angular gyrus, 4), 4 in the frontal cortex (left middle gyrus, 1; left inferior gyrus, 3), 2 in the left inferior temporal gyrus, and 1 in the left cerebellum. Although it is tempting to interpret the occurence of supra-threshold voxels in some of these regions, caution is dictated by the nature of the analysis, which involves a low (and non-directional) statistical threshold. What the analysis does clearly indicate is that the dual main effect obtained in the NAcc ROI analysis does not reflect an anatomically generalized, non-specific pattern. Whether the effect is unique to the NAcc or shared by other specific regions is a question that demands further experimentation.

ACC as a source of information on effort demands

As discussed earlier, effort discounting can be understood as a form of reference-dependent reward processing. Specifically, the cost of effort can be viewed as setting a reference point against which rewards are measured (see Figure 1). Thus, in considering the neural correlates of effort discounting, one important question is: How is the reference point set?

One neural structure that seems likely to be involved is the dorsal ACC. As previously noted, the ACC has been proposed to monitor task difficulty, possibly indexed by conflicts in information processing (Botvinick et al., 2004; Ridderinkhof et al., 2004). This perspective is consistent with our finding, in the present experiment, of greater ACC activation during task performance in high demand blocks than in low demand blocks. Other evidence suggests that the ACC may also play a role in effort-based decision making (Botvinick, 2007; Rushworth, Walton, Kennerley, & Bannerman, 2004; Walton et al., 2003). Furthermore, on an anatomical level, the ACC sends direct connections to the NAcc (Croxson et al., 2005; Kunishio & Haber, 1994). All of these considerations make it plausible that the ACC serves as a source of effort-demand information to the NAcc.

If this is accurate then, in the context of our experiment, one would expect activation in the ACC during the numeral classification task, within individual blocks, to correlate inversely with the NAcc response to subsequent reward cues. To test this prediction, we conducted a set of regression analyses based on estimates of block-by-block activation in ACC and NAcc (see Methods). A regression based on the entire group of experimental subjects confirmed a highly significant inverse relationship between ACC activation during task performance in individual blocks and the NAcc response to the subsequent reward cue (standardized regression coefficient (B), −0.20, p ≪ 0.0001).¹⁰ Notably, this relationship persisted even when an indicator variable for block type was included as a regressor (B = −0.16, p ≪ 0.0001), and when data for low and high demand blocks were analyzed separately (low demand: B = −0.10, p = 0.02; high demand: B = −0.22, p ≪ 0.0001). In other words, even after controlling for external demands for effort, activation in the ACC during effort expenditure predicted subsequent reductions in NAcc reward responses. This finding was found to be consistent across subjects, with subject-specific regressions yielding negative regression coefficients for ACC activation in 21 out of 23 cases (mean = −0.14, t(22) = −3.77, p = 0.001; Figure 3B).

In order to investigate the anatomical specificity of this relationship between ACC and NAcc, we performed the same regression analysis on activation in the left dorsolateral prefrontal cortex (DLPFC, see Methods), another region widely believed to play a role in cognitively demanding situations (Luks et al., 2002; Miller & Cohen, 2001), but not generally associated with a monitoring role (Botvinick et al., 1999). Although this region was indeed more active during task performance in high demand blocks than in low demand blocks (t(22) = 9.97, p ≪ 0.0001), its level of activation on individual blocks did not show any significant correlation with the NAcc response to the ensuing reward cue, either in an omnibus analysis (B = −0.01, p = 0.67) or in individual subject analyses (mean B = −0.004, t(22) = −0.14, p = 0.89; Figure 3B).

Effort discounting or reward prediction?

As noted under Methods, an important consideration affecting our experimental design was that NAcc responses to reward outcomes may be modulated by the degree to which such outcomes are predicted (Breiter et al., 2001; Knutson et al., 2005). A particular concern was that our subjects might spontaneously assume that $ outcomes would occur more frequently after high than low demand blocks. If subjects had adopted this assumption, this would have created an alternative explanation for our results, namely that NAcc responses were reduced after high demand blocks because subjects literally predicted a higher average level of reward in that context, as compared with low demand blocks. Note the critical difference between this and the effort discounting account, which explains the observed pattern not as a result of differential reward prediction, but as reflecting a process that values rewards relative to their associated effort demands.

In order to discourage differential reward prediction, we explicitly informed subjects that the frequency of $ outcomes would not differ between block types. The provision of these instructions militates, to some extent, against a reward prediction account of our findings. Nevertheless, it is admittedly conceivable that subjects elected to ignore the information we provided, as well as their own experience in the task, in order to believe that reward rates would differ between block types. In view of this possibility, it is worth considering two further pieces of evidence.

The first is our finding of a correlation between the magnitude of the NAcc effort discounting effect and ACC activity during task performance. In order to explain this aspect of the data in terms of reward prediction, one would need to assume that subjects not only predicted more frequent rewards on high demand blocks but, further, that the strength of this prediction was dynamically adjusted based on the specific level of difficulty encountered during performance of each individual block, as indexed by ACC activation. It seems unlikely that subjects made such fine-grained reward predictions.

The second piece of evidence involves an aspect of our data that has not been previously discussed. Recall that each block in our experiment began with the presentation of a visual cue that indicated whether the block would involve high or low demand (see Figure 2). Note that, for a subject who believed that $ outcomes were more likely on high demand blocks, these initial cues should convey differential reward predictions. Given previous evidence indicating that NAcc responds to events predictive of reward (see, e.g., Knutson & Cooper, 2005; Nicola, Yun, Wakabayshi, & Fields, 2004), one might therefore expect the cue leading high demand blocks to induce greater NAcc activation than the low demand cue. One problem with this prediction is that such a difference in cue-related activity might arise from sources other than reward prediction, such as arousal, motor preparation, or simple temporal overlap between cue- and task-related activation in NAcc. Fortunately, a cleaner prediction can be framed: If subjects expected more frequent $ cues on high demand blocks, then NAcc should respond more strongly to the high-demand cue than the low-demand cue, and because this difference would relate specifically to reward prediction, it should not be observed in the control group, where no rewards were delivered.

The pattern actually observed is presented in Figure 3c. NAcc did respond more strongly to the high-demand cue. However, contrary to a reward prediction account, this effect appeared with comparable magnitude in both the experimental and control groups. The difference was statistically significant in both the experimental group, t(22) = 4.615, p = 0.0001, and the control group, t(12) = 4.0474, p = 0.0016. An ANOVA with factors for cue type and group yielded a non-significant trend toward a main effect of group, F(68, 1) = 2.67, p = 0.12, but no sign of an interaction between cue and group, F(68, 1) = 0.04, p = 0.85.

While this pattern of NAcc responses is inconsistent with a reward prediction account, it is open to several other explanations. For example, as suggested above, it could be connected with elevated arousal in anticipation of high-demand blocks. On the other hand, it is inviting to consider that the NAcc response to the task cues might relate to the effort discounting effect seen later in each trial. As we have discussed, effort discounting can be viewed as an instance of reference-dependent reward processing. With this in mind, the cue-related NAcc response might be interpreted as reflecting a reference-setting event, with a stronger NAcc response reflecting the establishment of a higher reference point for reward. If this interpretation is valid, then one should expect to see a positive correlation, across experimental subjects, between the size of the cue-type effect in NAcc and the size of the effort-discounting effect. Such a correlation was indeed observed, r = 0.57, p = 0.005.¹¹

If the cue-related NAcc response does indeed represent a reference-setting event, it is interesting that this event occurs in the control condition, which involves no outcomes framed as rewards. This might indicate that the reference-setting event associated with anticipated effort is in some sense obligatory. Whatever its interpretation, this aspect of our findings seems to invite further experimental investigation.

Discussion

Taken together, our experimental results suggest that NAcc responses to cued rewards are decremented based on the effort required to earn them, a neural correlate of effort discounting. The data are also consistent with the idea that information about the demand for effort comes to the NAcc, at least in part and in some circumstances, from the dorsal ACC.

These conclusions are concordant with existing theories of both NAcc and ACC function. As noted earlier, the NAcc is widely understood to be involved in the processing of rewards (Breiter et al., 2001; McClure, York, & Montague, 2004; O’Doherty, 2004). One influential interpretation of existing findings is that the NAcc encodes errors in reward prediction (McClure et al., 2004; O’Doherty, Dayan, Friston, Critchley, & Dolan, 2003, but see Nicola, Yun, Wakabayashi et al., 2004). A critical feature of this account, also found in some contemporary behavioral theories of reward processing (Tversky & Kahneman, 1992), is that reward is represented not in absolute terms, but instead relative to a standard or reference-point. This reference-dependence accords well with the present findings, which suggest that the NAcc encodes a reward’s magnitude relative to its cost in effort.

Our results are also consistent with assertions that the NAcc plays a role in effort based decision-making (Salamone et al., 2003; Walton et al., 2006). This perspective focuses on cost-benefit analyses computed in advance of effort expenditure, while our focus —following recent work on delay discounting (Roesch et al., 2006) — has been on the moment when reward outcomes are revealed, following effort. Nevertheless, the ability to perform cost-benefit analyses depends on knowledge gleaned from previous experience with reward (McClure et al., 2004; O’Doherty et al., 2003) and effort (Botvinick, 2007) outcomes. Thus, there may be a functional connection between the pattern of NAcc behavior reported here and the proposed role of this structure in behavioral decision making. Of note, in preliminary work, Croxson, Walton and Rushworth (in press) have reported NAcc effort discounting effects analogous to those we have reported, but occurring when reward cues were presented prior to effort expenditure. Exploring the relationship between these findings and the ones reported here presents an important objective for further research.

As with NAcc, the present results also accord with previous work regarding the ACC. As noted above, one perspective on ACC function links it to the monitoring of task difficulty or the demand for cognitive control (Botvinick et al., 1999; Botvinick et al., 2001; Botvinick et al., 2004; Brown & Braver, 2005; Davis et al., 2005; Kerns et al., 2004; Ridderinkhof et al., 2004). The ACC has also been proposed to mediate in the subjective experience of mental effort (Naccache et al., 2005). Both of these perspectives are clearly consistent with our proposal that the ACC serves as a source of effort-demand information in the evaluation of rewards. According to another, not unrelated, theoretical perspective, the ACC has been portrayed as playing a key role in effort-based decision making (Botvinick, 2007; Floresco & Ghods-Sharifi, 2006; Rushworth et al., 2004; Walton et al., 2003). Walton and colleagues (1998) have proposed, in particular, that the ACC affects decision-making through its influence on NAcc. The resonance between this proposal and the work we have reported here is readily evident. Nevertheless, it is once again important to acknowledge the distinction between cost-benefit analyses computed ahead of action selection and the representation of rewards at the time of their conferral. It will be interesting to consider how the role of the ACC in modulating on-line reward processing, as studied in our experiment, may relate to its putative role in cost-benefit analysis and decision making.

While the present findings dovetail well with existing neuroscientific data, they also fit with behavior-based accounts of effort discounting. In research with both humans and animals, effort discounting has generally been inferred from a preference for rewards that can be easily obtained over rewards demanding greater effort (Denk et al., 2005; Floresco & Ghods-Sharifi, 2007; Kivetz, 2003; Phillips et al., 2007; Rudebeck et al., 2006; Rushworth et al., 2004; Salamone et al., 2007; Salamone et al., 2003; Schweimer & Hauber, 2006; Schweimer, Saft, & Hauber, 2005; Solomon, 1948; Walton et al., 2003; Walton, Bannerman, & Rushworth, 2002; Walton et al., 2006; Zipf, 1949). The present findings complement this behavioral pattern, providing evidence of a direct neural correlate for what has so far been primarily a theoretical construct motivated by behavioral observations.

Having made this point, it is also critical to acknowledge a small but important set of animal behavior studies that have been interpreted as running counter to the notion of effort discounting. For example, several studies (Clement, Feltus, Kaiser, & Zentall, 2000; Friedrich & Zentall, 2004; Kacelnik & Marsh, 2002) have shown that birds prefer rewards usually obtained through a high degree of effort over more easily obtained rewards, when free access is given to both (however, see Vasconcelos, Urcuioli, & Lionello-DeNolf, 2007). Understanding the relation between such findings and those we have reported here presents an important challenge. However, it should be noted that this challenge is not isolated to our own work. For it is not yet clear how the reported preference for effort-associated rewards fits with the general finding, from a much larger body of research, that animals tend to favor low-effort routes to reward. It is also worth noting that Clement and Zentall (2002) have demonstrated analogous effects when delay is substituted for effort, meaning that the interpretive challenge applies not only to effort discounting, but also to the even more firmly established construct of delay discounting.

The work we have reported raises a number of other issues for further investigation. For example, would the same pattern of NAcc activation hold in the case of physical, rather than mental, effort, and would the same relationship be observed between NAcc and ACC activation? Would the pattern change with alterations in NAcc dopamine concentrations, which have been shown to affect breakpoints in effort-based decision making (see Figure 1)? The experimental paradigm we employed might also be used to investigate individual differences relevant to effort and reward, including differences in intrinsic motivation, industriousness or need for cognition. Another dimension of interest from the point of view of individual differences is personal wealth, which has been shown to impact reward processing at both behavioral and neural levels (Tobler, Fletcher, Bullmore, & Schultz, 2007). It would be interesting to know if this factor also impacts effort discounting, as intuition suggests it might. Finally, it could be productive to investigate whether the relationship between effort and reward, as reflected in NAcc activation, is altered in clinical conditions characterized by changes in motivation, such as major depression and bipolar disorder, as some parallel research (e.g., Salamone, Correa, Mingote, Suzanne, & Farrar, 2006) has already begun to suggest.

Before closing, it is important to acknowledge some weaknesses of the present study that might be addressed in later experiments. First, although it was not statistically significant, a trend toward an overall difference in reaction times was observed between control and experimental groups. As discussed above, such a difference has limited impact on the interpretation of the data, since the difference in RT between high- and low-demand blocks was comparable between groups. Nonetheless, it would be desirable to see the central neuroimaging result replicated in the context of more precisely matched behavioral conditions. Second, the results of our whole-brain analyses indicated insufficient power to investigate the effort-discounting effect in a fully exploratory fashion. Methods for accessing this effect with greater statistical power would offer a clear advantage in further studying the relationship between effort and reward processing.