Choosing to Make an Effort: The Role of Striatum in Signaling Physical Effort of a Chosen Action

Abstract

The possibility that we will have to invest effort influences our future choice behavior. Indeed deciding whether an action is actually worth taking is a key element in the expression of human apathy or inertia. There is a well developed literature on brain activity related to the anticipation of effort, but how effort affects actual choice is less well understood. Furthermore, prior work is largely restricted to mental as opposed to physical effort or has confounded temporal with effortful costs. Here we investigated choice behavior and brain activity, using functional magnetic resonance imaging, in a study where healthy participants are required to make decisions between effortful gripping, where the factors of force (high and low) and reward (high and low) were varied, and a choice of merely holding a grip device for minimal monetary reward. Behaviorally, we show that force level influences the likelihood of choosing an effortful grip. We observed greater activity in the putamen when participants opt to grip an option with low effort compared with when they opt to grip an option with high effort. The results suggest that, over and above a nonspecific role in movement anticipation and salience, the putamen plays a crucial role in computations for choice that involves effort costs.

INTRODUCTION

The cost involved in an action is an important determinant in choice behavior (Kennerley et al. 2009). A number of animal and human experiments have examined how effort determines choice, and crucially, how the brain calculates and integrates effort into action value (Croxson et al. 2009; Floresco and Ghods-Sharifi 2007; Floresco et al. 2008; Kennerley et al. 2009; Rudebeck et al. 2008; Salamone et al. 1994; Walton et al. 2005, 2009). Other costs are better understood, for example the discounting of prospects the outcomes of which are accompanied by possible pain or loss (Seymour et al. 2007; Talmi et al. 2008, 2009) and by temporal delay (Kable and Glimcher 2008; McClure et al. 2007; Pine et al. 2009; Rudebeck et al. 2006; Stevens et al. 2005). Indeed temporal discounting may be confounded by effort discounting because effortful actions invariably involve greater time investment (but see Floresco et al. 2008 for effort discounting in rodents after controlling for time effects). Furthermore, effort can be expended as either mental work, for instance in performing complex cognitive calculations (Botvinick et al. 2009; Jansma et al. 2007), or physical work (Lewis 1964). The neurobiology of the latter remains relatively underexplored.

Animals constantly exploit their environment to minimize foraging costs and maximize reward (e.g., food) (Bautista et al. 2001; Kacelnik 1997; MacArthur and Pianka 1966). Effort may appear less dominant as a determinant of human behavior, but it remains the case that humans tend to favor one action over another if it involves less effort (e.g., taking a bus, instead of cycling to work). Indeed excessive effort discounting is likely to be a key marker of apathy, a function with significant societal and health impact. The neurobiology of pathological apathy, characterized by an inability to initiate simple day-to-day activities with excessive reliance on external control (a spectrum that incorporates abulia), is suggested by its strong association with damage to basal ganglia-prefrontal circuitry (Levy and Dubois 2006; van Reekum et al. 2005). Patients with a specific subtype of apathy, auto-activation deficit, are poor at converting basic reward valuation process into effortful action execution (Schmidt et al. 2008). In light of growing evidence for involvement of the striatum in value based decision-making, we hypothesized its involvement in action choice where a neural computation entails an integration of effort as a cost.

An extensive literature based on animal studies implicates regions such as nucleus accumbens (NAc) (Salamone et al. 2003) and anterior cingulate cortex (ACC) in effortful choice (Floresco and Ghods-Sharifi 2007; Rudebeck et al. 2006; Walton et al. 2009). By contrast, there are no human studies that examine the neural representation of physical effort to choose an action. In nonchoice contexts, the striatum and the ACC are activated when participants anticipate an upcoming action that entails effort (Croxson et al. 2009). More specifically, they found that activity in the striatum correlates with the anticipated effort for an action.

We designed a functional magnetic resonance imaging (fMRI) experiment to look for striatal involvement when humans make choices that entail physical effort. We employed a simple effort-based choice task where participants decide between holding a grip device and effortful gripping. The holding option entailed no effort and a minimal reward. The gripping option varied across two factors, namely monetary reward and force levels (percent of individual maximum force) indicated by a visual stimulus. In our imaging analysis, brain activity was time-locked to events at the time of choice, to index activity associated with, and effort modulation of, the decision to grip. We hypothesized activity in striatum would be associated with biasing choice away from actions that entail greater physical effort.

METHODS

General task description

The choice task was split in choice and execute periods. In choice periods, participants made a long series of consecutive choices between an effortful gripping option and a holding option that were indicated by visual stimuli (grip and hold stimuli; see Fig. 1). We manipulated levels of effort and reward in effortful gripping, and presented a fixed minimum reward with zero effort for the holding option; visually, effort levels were indicated by a vertical line, whereas reward levels were indicated by a horizontal line. In execute periods, participants executed a proportion of their selected options from the preceding choice period, by gripping (or simply holding) a hand device at the corresponding effort level to receive the corresponding reward amount.

Fig. 1.

Left top: grip and hold stimuli. Grip stimulus: a horizontal line indicated reward levels (in pence), a vertical line indicated effort levels (in % maximum grip). We added a random value to effort and reward levels of each grip stimulus; values in brackets show the averages. Hold stimulus: a horizontal line indicated a fixed reward value in pence. Middle: a schematic of the task. Choice period: in each choice trial, a fixation cross appeared, followed by a grip and a hold stimulus. Participants had to make a decision to grip or to hold. There were 12 choice trials; each grip stimulus was presented pseudorandomly. At the end of each choice period, the computer randomly selected 9 of 12 participants' choices from the preceding choice period to be executed. Execute period: immediately following the 12th choice trial, execute period comprised 9 trials; either grip or hold trial, is started. In the grip trials, a thermometer with a target level was displayed to guide squeezing the hand grip. In the hold trials, a frozen thermometer was presented. Each participant carried out 5 sets of choice and execute period in total. Bottom right: a thermometer stimulus is used to guide squeezing during execute period. The red “mercury” indicates current force level; yellow horizontal line indicates target level.

Participants

Eighteen right-handed participants [5 females, age: 27 ± 3 (SD) yr] were recruited through a university participant database. One participant was excluded from the analysis of brain activity due to excess motion artifact but was included in the behavioral analysis. All participants were paid £25–30 depending on duration of experiment they participated in. The study was approved by the University College London (UCL) ethics committee.

Stimuli and material

We refer to the stimuli potentially requiring effortful gripping as grip stimuli; and the stimulus requiring noneffortful holding of the hand-grip device as the hold stimulus. As with the visual stimuli used in Croxson et al. (2009), grip stimuli comprised red circles with two black lines (see Fig. 1). Vertical lines indicated effort with two levels [left is low effort (LE), right is high effort (HE)], while horizontal lines indicated reward levels [bottom is low reward (LR), top is high reward (HR)]. To avoid boredom, we varied the effort and reward level trial by trial by adding a pseudo-random value to base values of effort (40% maximum force for LE and 85% for HE) and reward (3 pence for LR and 11 pence for HR) of each stimulus. These values were drawn from a normal distribution with 0 ± 1 SD and ranged from −5.2 to 5.4% maximum force for effort and −2.6 to 2.7 pence for reward. The hold stimulus is a red circle with a horizontal line on the bottom representing a fixed low reward (2 pence) and no vertical line.

For cue that guides squeezing, we constructed an image of a white thermometer with a yellow horizontal line indicating the target force level, set at 85 or 40% of thermometer height according to choice of high and low effort, respectively, and red “mercury” that moves vertically as participants squeeze the hand-grip device (Fig. 1). This thermometer cue was presented after a choice of a grip stimulus. When participants chose a hold stimulus, on the other hand, a “frozen” thermometer was presented as participants simply held the hand-grip device.

Participants executed their choices by squeezing (or holding), with their right hand, a hand-grip device molded from two plastic cylinders that compressed an air tube (Pessiglione et al. 2007; Talmi et al. 2008) connected to a transducer (Honeywell, Morristown, NJ) that converted air pressure into a voltage output. Thus variation in air compression within the cylinders due to the force applied resulted in different voltage signals that are linearly proportional to exerted grip force. The signal was recorded (Spike2, Cambridge Electronic Design) and transmitted to Matlab 6.5 (www.mathworks.com). The force signal was translated as red mercury level in the thermometer (Fig. 1) as a veridical, real-time visual feedback for squeezing. All stimuli were presented using Cogent 2000 (//www.fil.ion.ucl.ac.uk/ and http://www.icn.ucl.ac.uk/) and Cogent Graphics (John Romaya at the Laboratory of Neurobiology at the Wellcome Trust Centre for Neuroimaging, at UCL).

Procedure

Participants lay on the scanner bed to undergo, successively, force calibration, training, four experimental blocks and a final structural scan. Calibration and training blocks were completed as participants lay on the scanner bed outside the magnet, while experimental blocks and the structural scan were completed as participants lay inside the magnet. Participants completed postscan questionnaires outside the scanner at end of experiment.

FORCE CALIBRATION.

With the participant reclined, we measured the maximum voluntary force three times and calibrated grip levels as the highest value individually. The maximum voluntary force was also measured with the participant in a reclined position just after the experimental task, and we confirmed that there was no significant difference between them [t(17) = 0.68, P = 0.50]. This result suggests that choices in the task were not influenced by fatigue or nonspecific time effects.

TRAINING BLOCK.

Following force calibration, participants completed 86 single-stimulus training trials that comprised a cue presentation, a button press, a thermometer presentation as a prompt for squeezing, and a reward outcome presentation. At the beginning of each training trial, either a hold or grip stimulus was randomly presented until participants made a button press with their left hand. Following a hold stimulus, a frozen thermometer was presented for 6 s independent of the participants' grip force (participants typically just held the hand-grip device). By contrast, following a grip stimulus, a thermometer was presented for 6 s and participants squeezed the hand-grip device to reach the target until the thermometer disappeared. Then participants saw the reward outcome, either 2 pence for a hold trial or the reward corresponding to the horizontal line on the stimulus for a grip trial. For the grip trials, if participants did not reach the target within 2 s after thermometer onset or if they released the hand-grip before 6 s expired, the reward outcome was 0 pence, and the trial was aborted. Participants were not informed about the precise effort and reward amounts but learned the stimulus-effort-reward contingencies from experience (Hertwig et al. 2004).

BEHAVIORAL CHOICE TASK.

After the training block the participants were moved into the magnet to perform the choice task of four sessions with a rest period (≤3 min) between the sessions.

We split the task into choice and execute periods to remove motor preparatory brain activity in anticipation of gripping that is likely to occur during choice. In the choice period, participants made 12 consecutive choices between a grip and a hold stimulus by emitting a left or right button press. Then in the execute period, they executed nine trials, which are randomly selected from the 12 choices chosen by the participant in the previous choice period, by squeezing (or just holding) the hand-grip. At the beginning of each period, a visual message was displayed to indicate the start of the relevant period (choice or execute). A pair of the choice and the execute period was repeated five times in each session (see Fig. 1).

At the beginning of each choice trial, a fixation cross appeared for 200 ms, followed by a grip and a hold stimulus on either side. The participant chose one of the two by pressing button corresponding to the position on the screen within 1,800 ms. There were four types of grip stimulus with two reward and two effort levels, the order of presentation of each grip stimulus was pseudorandomized. Following this, the stimuli disappeared, and a fixation cross re-appeared after 500 ms, indicating the start of next choice trial. After 12 consecutive trials completed, an execute period commenced. At the beginning of each execute trial, a fixation cross appeared for 200 ms. If the execution trial was a grip execution, a message prompted participants to get ready to grip, and then a thermometer was presented to guide squeezing for 6 s. The trial aborted if participants did not reach the target within 2 s after thermometer onset or if they released the hand-grip before 6 s expired with a reward outcome of 0 pence. On average, 2 ± 0.7% of all trials were aborted; these trials were included in the analysis. To reduce noise caused by no-go signal during a hold execution trial, participants did not see any prompt message and instead were immediately presented with a frozen thermometer for 6 s. Overall, participants completed 240 choice trials and 180 execute trials, split in four sessions of 60 choice trials and 45 execute trials.

POSTSCAN QUESTIONNAIRES.

Immediately after the scanning session, participants completed a 20-item persistence scale that measures individual propensity to work harder when facing daily challenges (e.g., “I usually push myself harder than most people do”) (Cloninger et al. 1993) on a computer outside the scanner. Participants also responded to two manipulation check questions for reward and effort (see methodological details in supplemental material)¹ on paper. To check that participants understood the reward amount indicated by the stimuli, participants were shown a red circle with a horizontal line and responded to the question “how much money does the horizontal line on the circle mean?” Participants did this twice, one for a red circle with the line at the top (high reward) and one at the bottom (low reward). To test that participants perceived low and high effort levels differently, participants were presented with the thermometer and responded to the question “how much money do you think is considered a fair pay for gripping at the yellow line 10 times in a row?” Participants did this twice, one when the target line of the thermometer was at the 40% (low effort) and 85% (high effort) of the height of the thermometer. Responses to the reward item showed the desired effect: participants estimated the amount of reward for stimuli indicating high reward and low reward reasonably accurately (HR = 9.61 ± 0.36; LR = 2.1 ± 0.27 pence), and the difference was significant t(17) = 18.93, P = 0.00001]. Likewise, our effort manipulation check also showed that high and low effort levels were perceived differently: the estimate for an expected fair pay to squeeze ten-times at high effort (£2.39 ± 0.69) was significantly greater than the estimate for low effort [£0.89 ± 0.26; t(17) = 2.80, P = 0.012].

Image acquisition

We used a 3T Siemens TRIO system (Siemens, Erlangen, Germany) with 12-channel head coil to acquire both T1-weighted anatomical images and T2*-weighted MRI transverse echoplanar images (EPIs; 64 × 64 mm, TR/TE = 2.72 s/30 ms) with BOLD contrast. The EPI sequence was optimized for maximizing signal in inferior brain regions (Weiskopf et al. 2006). Each EPI comprised 40 3-mm-thick contiguous axial slices taken every 3 mm, positioned to cover the whole orbitofrontal cortex, striatum, up to the anterior cingulate and motor cortices. In total, 180–212 volumes were acquired for each participant in one session. The first five volumes were discarded to allow for T1 equilibration effects. The field maps were acquired between the second and third scanning sessions. For the structural images, we acquired a standard high-resolution T1-weighted anatomical image with acquisition matrix 256x240, TR/TE/flip angle = 7.92 ms/2.48 ms/16°, voxel size 1 × 1 × 1 mm, 176 axial slices (Deichmann et al. 2004).

Imaging analysis

Data were analyzed using Statistical Parametric Mapping (SPM8b; Wellcome Trust Centre for Neuroimaging, London, UK, http://www.fil.ion.ucl.ac.uk/spm). Five preprocessing steps involved intramodal realignment and unwarping, intermodal co-registration, segmentation, normalization, and smoothing. First, we re-aligned all EPI volumes to the first volume to correct for inter-scan movement. Images were unwarped using field maps to remove unwanted gripping-related variance without removing variance attributable to the motor task (Anderson et al. 2001). Second, the mean motion-corrected image was co-registered to individual's T1 images using a 12-parameter affine transformation. To correct for different acquisition times, the signal measured in each slice was shifted relative to the acquisition of the lower slice using sinc interpolation in time. Third, individual T1 images were segmented based on gray and white matter, a method fairly robust and accurate in creating spatial normalization parameters for the EPI and anatomical images (Ashburner and Friston 2004). Fourth, the co-registered EPI and T1 volumes were normalized using segmentation parameters, based on the Montreal Neurological Institute (MNI) reference brain in Talairach space (Talairach and Tournoux 1988) and re-sampled to 3 × 3 × 3 mm³ and 1 × 1 × 1 mm³ voxels, respectively. Finally, we smoothed all normalized images with an isotropic 8-mm full-width half-maximum Gaussian kernel to account for intersubject differences and allow valid statistical inference according to Gaussian random field theory (Friston et al. 1995a,b). The time series in each voxel were high-pass filtered at 1/128 Hz to remove low-frequency confounds and scaled to a grand mean of 100 over voxels and scans within each session.

We performed a random-effect, event-related, statistical analysis. First, we specified a separate general linear model (GLM) for each participant by creating separate regressors for each of the four scanning sessions. To highlight activity correlating with anticipated effort and with the choice to grip or hold, we defined four regressors-of-interest representing four event types that varied in effort level and participants' choice (low effort vs. high effort and grip vs. hold) at choice onset: grip-low effort (gripLE), grip-high effort (gripHE), hold-low effort (holdLE), and hold-high effort (holdHE). Furthermore, to assess activity correlating with reward, we entered a trial-by-trial reward value (3 or 11 pence ± a random value) as a parametric modulator for each of the four regressors. We entered two regressors-of-no interest from the grip and hold trials in the execute periods at thermometer onset with 6-s duration; suprathreshold activity for grip > hold contrast in execute periods is found in left primary motor cortex (see Additional results in supplemental materials). We convolved each regressor with a canonical hemodynamic response function and its temporal derivatives. Motion parameters from preprocessing were entered into the design matrix.

We computed a set of contrasts for each participant testing the main effects of choice, effort, and the interaction. Consistency across the resulting maps of sensitivity for each participant was tested in a series of one-sample t-test as group analyses. As we found persistence correlated with behavioral choice (reported in the following text), we entered persistence score as a covariate at the second level and ran a whole-brain analysis, thresholded at P = 0.001 uncorrected, >5 voxels, to search for areas active in response to choice (grip vs. hold), effort (low vs. high), choice-effort interaction, and simple effects of effort at both choices (gripHE vs. gripLE and holdHE vs. holdLE).

RESULTS

We sought to investigate the influence of effort on behavioral measures and brain regions of which activity is involved in biasing choice from options that involved greater effort.

Behavioral results

CHOICE.

Participants chose to grip more often when the effortful option entailed low effort than high and when the effortful option offered high reward than low (Fig. 2A). Main effects of effort and reward were significant, F(1,17) = 13.07, P = 0.002, partial eta squared 43%; F(1,17) = 105.08, P < 0.0001, partial eta squared 86%, respectively. There was no significant interaction between them.

Fig. 2.

Behavioral choice, subjective rating, and response times (RTs). A: proportion of trials where participants chose to grip (dark shade) and their subjective rating (light shade) for each option. Participants chose to grip more often when the reward offered was high than when it was low and when the effort anticipated was low than when it was high. The interaction was nonsignificant. Liking (light shade) was higher for options with high reward than for options with low reward, higher for options with low effort than for options with high effort, and comparable between hold and low effort-low reward. The interaction was nonsignificant. B: the same liking data to A showing that on average, the order of rating from lowest to highest is high effort-low reward, low effort-low reward, high effort-high reward, and low effort-high reward. C: RTs were slower for choice to hold than for choice to grip. (mean ± SD).

SUBJECTIVE RATING.

As seen in Fig. 2A (dark shade), participants rated the hold and low effort-low reward option comparably. We compared the liking for hold (as baseline) and each of the grip options for each participant and found that the difference in subjective liking was significantly higher for high reward than low reward [F(1,17) = 173.41, P < 0.0001, partial eta squared 91%] and for low effort than high effort [F(1,17) = 86.61, P < 0.00001, partial eta squared 83%]. The interaction was not significant. This result suggests that effort and reward influenced likeability of an option. Based on the group-averaged liking scores, we could then describe the order of subjective liking for these actions, from lowest to highest: high effort-low reward, low effort-low reward, high effort-high reward, and low effort-high reward (see Fig. 2B). These findings suggest a fair generalizability to common views on effortful and rewarding actions whereby actions with more effort and less reward are less liked.

RESPONSE TIMES.

Overall, the participants took significantly longer time in choosing to grip than to hold [t(17) = 28.95, P < 0.0001; Fig. 2C]. We ran a separate ANOVA to formally test the effects of effort and reward on response times (RTs). A 2 × 2 (effort × reward) ANOVA revealed that regardless of choice (grip/ hold), RTs were slower for low (M = 994 ± 25 ms) than high reward (M = 764 ± 60 ms), F(1,17) = 566.59, P < .0001, partial eta squared 97%; and for high (M = 882 ± 126 ms) than low effort (M = 876 ± 125 ms), F(1,17) = 4.61, P < 0.046, partial eta squared 21%. There was no significant interaction. Participants represented each option by taking account of both its effort and reward. We ran a separate imaging analysis with RTs as a covariate of-no-interest at the first level analysis, and this analysis did not change our main findings reported below.

PERSISTENCE.

A persistence trait is linked to self-directedness (Cloninger et al. 1993), a characteristic especially lacking when an individual suffers from apathy. We calculated a correlation between persistence scores with the effects of effort, reward, and interaction on choice. We found that the main effect of effort on choice, regardless of reward level, was negatively correlated with persistence, r = 0.59, r² = 34%, P = 0.01. As persistence score decreases, there was a greater difference between choice to grip an option with low effort compared with choice to grip an option with high effort: i.e., less persistent participants much preferred low compared with high effort, while those with high persistence (or less apathy) chose to grip options with low and high effort equally often (Fig. 3A). Other correlations with reward and interaction effects on choice were nonsignificant.

Fig. 3.

Persistence, behavioral choice, and in dorsal anterior cingulate cortex signal. A: persistence is negatively correlated with the effect of effort on choice (n = 18). Regardless of reward, low persistence is associated with a higher preference for options with low effort, whereas high persistence is associated with indifference between options with low effort and options with high effort. B: activity in the dorsal anterior cingulate cortex when the rejected option entailed low effort is positively correlated with persistence (P < .001 uncorrected, 11 voxels; n = 17).

fMRI results

To extend our behavioral finding that effort influenced choice, we examined BOLD response when participants chose to grip or to hold and when the required effort was high or low. We added trial-by-trial reward level as a parametric modulator for each regressor and persistence score as a subject-by-subject parametric regressor at second level.

Choice-related activity

The main effect of choice (choice to grip > choice to hold) was associated with activity in the anterior part of right superior frontal gyrus (Z = 3.49, x = 18, y = 53, z = −2, 7 voxels; Table 1). We did not find any suprathreshold activity for choice to hold > choice to grip. No suprathreshold clusters were evident for the main effect of effort or interaction between choice effort.

Table 1.

MNI coordinates and choice

		Coordinates, mm
Region	Nearest Brodmann Areas	x	y	z	Z	No. of Voxels	P
Contrast: Choice to Grip > Choice to Hold
Superior frontal gyrus	10	+18	+53	−2	3.49	7	0.0001 (unc.)
Middle parietal lobe	7, 19	+21	−52	+25	3.32	5	0.0001 (unc.)

Montreal Neurological Institute (MNI) coordinates of regions the activity of which is correlated with choice (thresholded at P = 0.001, uncorrected >5 voxels). unc, uncorrected.

Effort-related activity

We next explored activity modulated by effort level for trials where participants chose to grip, chose grip trials, and for trials where participants chose to hold, chose hold trials, separately. Particularly, using a whole-brain analysis, we tested for striatal activity associated with effort information of the option. We also explored contrasts that were modulated by persistence trait.

For the chose grip trials, we observed significant striatal activity when participants chose to grip a low compared with when they chose to grip a high effort option (gripLE > gripHE; Fig. 4). This activity extended dorsally toward the caudate with a peak in the left putamen (Z = 4.04, x = −27, y = 8, z = 4, 51 voxels) and survived a more stringent threshold (cluster corrected family-wise error P = 0.01). Regardless of reward level, the dorsal aspect of the putamen signaled effort information of the chosen action, with lower effort invoking greater signal. In the same contrast, we also found activity in the left motor cortex (Z = 3.6, x = −33, y = −19, z = 49, 32 voxels), right cingulate motor area (Vogt 2005) (Z = 3.51, x = 12, y = −28, z = 46, 6 voxels), and right supplementary motor area (SMA, Z = 3.31, x = 3, y = −16, z = 55, 8 voxels). The reverse contrast (gripHE > gripLE) did not show any suprathreshold activity. The statistics of the activations are summarized in Table 2.

Fig. 4.

Activity in left putamen is higher when participants chose to grip an option which involved low effort than when they chose to grip an option which involved high effort (cluster-corrected family-wise error P = 0.01, 51 voxels). Bar graph depicts the parameter estimates for this contrast for visual illustration.

Table 2.

MNI coordinates and effort

		Coordinates, mm
Region	Nearest Brodmann Areas	x	y	z	Z	No. of Voxels	P
Contrast: GripLE > GripHE
Putamen	N/A	−27	+8	+4	4.04	51	0.01 (corr.)
Putamen	N/A	−21	+20	−2	3.81
Primary somatosensory cortex	1	−57	−19	+43	3.64	13	0.0001 (unc.)
Primary motor cortex	4p	−33	−19	+49	3.60	32	0.0001 (unc.)
Primary somatosensory cortex	3b	−42	−25	+49	3.32	32	0.0001 (unc.)
Cingulate motor area	23, 24	+12	−28	+46	3.51	6	0.0001 (unc.)
Supplementary motor area	6, 4a	+3	−16	+55	3.31	8	0.0001 (unc.)
Supramarginal gyrus	7, 40	−51	−40	+34	3.61	12	0.0001 (unc.)
Supramarginal gyrus	7, 40	−45	−43	+28	3.34		0.0001 (unc.)
Middle temporal gyrus	39	−57	−52	+19	3.36	8	0.0001 (unc.)
Contrast: HoldHE > HoldLE
Mid-brain	N/A	+9	−25	−8	3.57	7	0.0001 (unc.)
Putamen	N/A	−33	−13	−5	3.39	5
Middle Temporal Gyrus	37	+60	−34	−8	3.32	6
Contrast: Persistence × HoldLE
Anterior cingulate cortex	24	+3	+26	+25	3.70	11	0.0001 (unc.)
Posterior part of Middle Temporal gyrus	19	+51	−76	+13	3.40	8	0.0001 (unc.)

MNI coordinates of regions the activity of which is correlated with effort (thresholded at P = 0.001, uncorrected > voxels). unc, uncorrected.

For the chose hold trials, on the other hand, we did not find any suprathreshold activity with a contrast of trials where the rejected option involved low or high effort (holdLE > holdHE). The reverse contrast (holdHE > holdLE) yielded an enhanced activity in midbrain, in the vicinity of ventral thalamus (Z = 3.57, x = 9, y = −25, z = −8, 7 voxels; Table 2) for rejecting options with high effort compared with rejecting options with low effort.

Finally, we tested if persistence modulates effort-related activity, using the behavioral persistence scale as a covariate. We found no effect on activity associated with effort-related choices to grip but found that a persistence trait significantly modulated activity in right dorsal ACC when participants rejected an option with low effort (Z = 3.7, x = 3, y = 26, z = 25, 11 voxels; Table 2). Thus the more persistent a subject is, the greater the activation in dorsal ACC when rejecting an option that entailed low effort (Fig. 3B). This was the only significant correlation between persistence and the BOLD response to each condition.

Activity reflecting reward modulation

With reward level as a parametric modulator, we found a significant correlation with activity in the SMA (Z = 3.61, x = −3, y = −19, z = 55, 8 voxels; Table 3) for the contrast chose grip > chose hold trials. We also tested for reward modulation in other contrasts and did not find any suprathreshold activity. However, driven by a strong prediction that nucleus accumbens (NAc) may be involved in reward processing (Knutson et al. 2005), we lowered the threshold to P < 0.005 (uncorrected, >5 voxels) and found a small but significant cluster at the vicinity of NAc (Z = 2.84, x = 0, y = −11, z = 11, 5 voxels) that positively correlated with reward only in trials where they opted an option with high effort (Fig. 5).

Table 3.

MNI coordinates and reward

		Coordinates, mm
Region	Nearest Brodmann Areas	x	y	z	Z	No. of Voxels	P
Contrast: reward × choice to grip > choice to hold
Inferior temporal gyrus	37	−51	−58	−5	4.07	10	0.0001 (unc.)
Supplementary motor area	6	−3	−19	+55	3.61	8	0.0001 (unc.)
Contrast: Reward × GripHE
Nucleus accumbens	N/A	0	+11	−11	2.84	5	0.002 (unc.)

MNI coordinates of regions the activity of which is correlated with reward (thresholded at P = 0.001, uncorrected > 5 voxels except for the last contrast; thresholded at P = 0.005, uncorrected > 5 voxels). unc, uncorrected.

Fig. 5.

Reward level is positively correlated with activity in bilateral nucleus accumbens when participants chose to grip an option that involved high effort. Activation displayed in pink is thresholded at P < 0.005 (uncorrected, 5 voxels), activation displayed in yellow is thresholded at P < 0.05 (uncorrected, 582 voxels).

DISCUSSION

The present study investigated behavioral and brain activations involved in choosing an action based on physical effort. Behaviorally, we show that effort acts to discount the value of an action, an effect reflected in lower ratings and lower preference for options with high effort. This finding supports previous laboratory and field experiments with animals, highlighting a sensitivity to action costs including higher fixed reinforcement schedule in lever presses, weight of the levers, higher metabolic requirements, longer traveling distance in foraging, and a higher physical response requirement of climbing when compared with walking (Eisenberger et al. 1989; Marsh et al. 2004; Salamone et al. 1994; Stevens et al. 2005; Walton et al. 2006, 2009). Our behavioral finding also accord with a human observation study of pedestrian walking efficiency (Bitgood and Dukes 2006).

There are two novel features in our task. First, unlike previous work in healthy humans, we employed physical effort rather than mental effort. Studying how effort is conceptualized in other fields, namely clinical neurology and behavior ecology, allowed us to determine the likely critical variables in relation to how effort influences behavior. Autoactivation deficit, the most severe form of apathy, quantitatively reduces the initiation and execution of actions and this contrasts with a “cognitive inertia” observed in less severe forms of apathy (Levy and Dubois 2006). A foraging literature in animals is concerned with the computation of physical effort costs such as metabolic rates (e.g., Marsh et al. 2004) in determining choice of foraging methods (e.g., walking or flying). These sets of observation provided us with a strong motivation for manipulating physical rather than mental effort. Second, in daily life, expending more effort often requires more time. While it is often experimentally difficult to disentangle the two, we were able to examine effort costs while controlling for time effects by equating the grip duration in high, low, and no effort conditions.

We found an association between persistence and the effect of effort on choice, which suggests our task captures a tendency to persist in everyday tasks, thus strengthening the interpretability and generalizability. Although this correlation could be driven by other traits such as obedience to experimenter or social desirability, there are good reasons to think otherwise. In our task, participants knew that the experimenters could not see their actual choice during the experiment, and this is likely to eliminate desirability biases. Moreover, the correlation with persistence was selective to the effect of effort, not to reward effect, nor did it correlate with effort-reward interaction. Nevertheless the generalizability of our task is subject to further testing.

Overall, choice and reward recruited frontal circuitries. We observed a modulation of activity for reward in the SMA when participants opted to grip regardless of actual effort levels. SMA region has been previously implicated in movement planning (Shima and Tanji 1998), which suggests that the choice to grip may evoke a representation of the outcome of the chosen action, which in these instances is correlated with reward expectation. Croxson et al. (2009) identified activity in the striatum, including the putamen, corresponding with net value (cost in terms of time and effort divided by reward) of an upcoming action. This led us to hypothesize involvement of striatum in effort-based choices in humans. We designed our experiment such that motor preparatory activity did not contaminate BOLD response during choice events (see Fig. 1). Notably, we found that the putamen was more active during anticipation of low relative to high effort, a finding that argues against traditional notions of the putamen being solely involved in pure motoric aspects of movement execution (e.g., Marchand et al. 2008; Prodoehl et al. 2009) and points to a role in a higher order aspect of action valuation (Tobler et al. 2007) that in this study pertains to a consideration of effort cost.

Previous rodent studies provide evidence for involvement of nucleus accumbens (ventral striatum) in effort-related responses (Salamone et al. 2007). A direct comparison of the regional anatomy of the striatum is difficult between humans and rodents. In humans there is good evidence of anatomical and functional dissociation between dorsal (dorsal caudate-putamen) and ventral (nucleus accumbens, ventral putamen/caudate and olfactory tubercle) (e.g., O'Doherty et al. 2004), but the connectivity of dorsal and ventral striatum share a similar parallel organization (Haber et al. 2000) The dorsal striatum has a stronger role in action learning and choice (as compared with passive prediction), which is the central way in which effort impacts on behavior in our task. Croxson and co-workers (2009) found a large cluster of activation spanning across the dorsolateral and ventromedial aspects of the striatum that correlated with the net value of an upcoming action, consistent with the notion that broad regions of the striatum may be sensitive to the cost of an action. Our finding of involvement of putamen along with previous work provides converging evidence that the striatum is implicated in effort-related choices in human and across species.

An important caveat to our interpretation of putamen activity as related to economic cost is that we do not see positive activity related to financial reward per se in this region. First, our imaging analysis was not designed to assess a simple difference in activity for high versus low reward. Second, we failed to identify a significant modulation of reward in our effort contrasts despite many previous demonstrations elsewhere for reward-related activity in this region (Croxson et al. 2009; Knutson et al. 2005; Pessiglione et al. 2007; Schmidt et al. 2009). One possible, and intriguing, explanation for this failure is that it may relate to a relative lack of salience of reward, as compared with effort, in the task. Even so, high reward still had a strong effect on behavior in the task and modulated brain activity for other contrasts in the SMA and in the NAc.

We expected to observe involvement of ACC in effortful behavior. For example, rodent experiments report that rats that expend effort for a larger gain preoperatively, choose an effortless, small reward, after a lesion in the ACC (Floresco and Ghods-Sharifi 2007; Walton et al. 2002, 2009). Monkey single-cell recordings (Kennerley et al. 2009), and human imaging experiments with passive action valuation (Croxson et al. 2009) or mental load (Botvinick et al. 2009) also report enhanced ACC activity with increasing effort. Nevertheless, a closer look at previous work on effort points to important differences in our paradigm that explain the absence of a high > low effort or low > high effort signal in the ACC. There has been no previous work that specifically examined an active evaluation of physical effort from human participants, and this is an important feature that our study has addressed.

Repetitive responding is another dimension of physical effort, which, like force production, is differentially influenced by dopaminergic manipulation (Ishiwari et al. 2004). Both aspects of effort may be associated with a behavioral trait of persistence that characterizes a human tendency to exert self-regulatory effort (Segerstrom and Nes 2007) to achieve long-term goals (Duckworth et al. 2007). In our task, we varied force production alone and found that persistence was associated with activity in dorsal ACC when participants rejected an option with low effort. This provides provisional support for an extensive neurological literature that links circuitry damage involving the ACC to various motivational impairments, as for example seen in apathetic patients (Eslinger and Damasio 1985; van Reekum et al. 2005). An important future research avenue would be to examine if repetitive responding interacts with force production in influencing action choices and how this relates to a persistence trait and apathetic syndromes as seen in human pathology.

Choosing to make a physical effort in our study reflects a critical evaluation of whether an action is worth taking, a pertinent cognitive process that may be lacking in dopamine-depleted conditions such as Parkinson's disease and apathy. Indeed evidence in rodents suggests that dopamine antagonism biases preference away from expending effort for a larger gain after controlling for time effects (Floresco et al. 2008). This evaluation also captures an individual propensity to persist through daily challenges. Vital research avenues in the future might fruitfully test the modulatory role of dopamine in effort expenditure and explore different degrees of cost-evaluative decisions made by healthy individuals, apathetic patients, or a subpopulation characterized by low persistence trait.

GRANTS

This work was supported by a Wellcome Trust Programme Grant to R. J. Dolan. N. Chater is supported by a Leverhulme Trust Major Research Fellowship and the Economic Learning and Social Evolution Research Centre at University College London.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).