Abstract
The ability to perceive and exercise control over an outcome is both desirable and beneficial to our well-being. It has been shown that animals and humans alike exhibit behavioral bias towards seeking control and that such bias recruits the ventromedial prefrontal cortex (vmPFC) and striatum. Yet, this bias remains to be quantitatively captured and studied neurally. Here, we employed a behavioral task to measure the preference for control and characterize its neural underpinnings. Participants made a series of binary choices between having control and no-control over a game for monetary reward. The mere presence of the control option evoked activity in the ventral striatum. Importantly, we manipulated the expected value (EV) of each choice pair to extract the pairing where participants were equally likely to choose either option. The difference in EV between the options at this point of equivalence was inferred as the subjective value of control. Strikingly, perceiving control inflated the reward value of the associated option by 30% and this value inflation was tracked by the vmPFC. Altogether, these results capture the subjective value of perceived control inherent in decision making and highlight the role of corticostriatal circuitry in the perception of control.
Keywords: corticostriatal circuitry, perceived control, striatum, subjective value, vmPFC
Introduction
Our sense of control over an outcome hinges on our perceived ability to manipulate and influence the environment to our advantage. While the ability to exercise real objective control over an outcome can be behaviorally reinforcing, it is the perception or subjective belief in having control that serves a basic need and contributes to our general well-being in 2 important ways (White 1959). First, it has been demonstrated in both animals and humans alike that the perception of control has protective effects to blunt external stressors and can dampen depressive symptoms such as anxiety, passivity, and helplessness (Thornton and Jacobs 1971; Maier and Seligman 1976; Abramson et al. 1978). Second, fulfilling the sense of control can be rewarding in and of itself, suggesting that perceived control generates positive affect that can bias behaviors accordingly (Leotti and Delgado 2011, 2014). Taken in conjunction with the pervasive manifestation of loss of control in psychopathologies (Glass and McKnight 1996; Frazier et al. 2004; Bechara 2005), the significance of perceiving control as both desirable and valuable to an organism is notable.
From an evolutionary perspective, several prominent theories have proposed that organisms have an inherent need for control that bias them towards environments conferring the perception of control (Rotter 1966; Bandura 1977; Ajzen 1991). This is supported by the observation that organisms across species show a clear preference to perform control-seeking behaviors (Catania and Sagvolden 1980; Suzuki 1997, 1999; Bown et al. 2003). One idea is that this preference for having the option to exert control is manifested as an affective signal that is processed in the brain’s reward system (for review see Ly et al. 2019). Using choice as a proxy for control, for example, neuroimaging studies have reported that participants had greater ventral striatum activation in response to cues that were associated with an opportunity for choice compared with cues associated with no choice opportunity (Leotti and Delgado 2011, 2014; Fujiwara et al. 2013). The presence of controllability has also been linked to dopamine release in the nucleus accumbens (NAcc), providing a potential molecular-level account of perceived control and substantiating the observation of NAcc activation in neuroimaging experiments (Cabib and Puglisi-Allegra 2012; Cockburn et al. 2014; Ikemoto et al. 2015). Another complimentary idea is that the preference for control can help cope with external stressors, which is consistent with the theory of “learned helplessness” (for review, see Maier and Seligman 2016). This line of work has implicated the ventromedial prefrontal cortex (vmPFC) as the neural substrate for detecting control and mediating the protective effects of control in response to external stressors (Amat et al. 2005; Maier et al. 2006; Maier and Watkins 2010).
Collectively, the aforementioned findings suggest that experimental conditions emphasizing a sense of perceived control over potential outcomes is not only desirable but also associated with regions involved in affective processing such as the striatum and the vmPFC (Delgado 2007; Haber and Knutson 2010; Bartra et al. 2013). An intriguing question is whether perceived control itself carries a subjective value that changes how the potential reward is processed and in turn influences reward-seeking behaviors. Here, we test the possibility that the desirable quality of perceived control could artificially inflate the subjective value of the actual reward and trigger approach behavior, even to the extent of incurring a cost to have control—that is, choosing a reward with an objectively smaller expected value (EV).
In this paper, we implemented a 2-alternative choice task to isolate the subjective value of control and study its neural correlates. Briefly, while undergoing functional magnetic resonance imaging (fMRI), human participants were instructed to make a series of binary choices between an option conferring behavioral control and another that relinquished control. By manipulating the reward magnitude for each choice pair and examining participants’ choice patterns, we derived a subjective value for control and investigated its neural underpinnings. We hypothesized that participants would show behavioral bias towards exercising control and this preference would recruit regions such as the striatum and vmPFC.
Methods
Participants
A total of 31 right-handed individuals (11 males and 20 females) between the ages of 18 and 37 (mean [M] = 23.3, standard deviation [SD] = 5.1) were recruited from the Rutgers University community for this study (see Supplementary Material for details on sample size determination). Participants were prescreened for any history of psychiatric and neurological illness. Participants were given monetary compensation for their voluntary participation in the experiment. In addition, they could also earn up to $20 of bonus monetary reward based on task performance. All participants provided written informed consent in accordance with the experimental protocol approved by the Rutgers University Institutional Review Board. One participant did not complete the experiment due to equipment failure and was excluded from subsequent behavioral and neural analyses. Three additional participants completed the experiment but were excluded from subsequent analyses due to complications during scanning session (e.g., participants closed eyes in scanner or did not follow directions). Final data analysis was conducted on 27 participants (9 males and 18 females; M = 22.4, SD = 4.3).
Experimental Design
The goal of the experiment was to quantify the behavioral and neural substrates of how much participants valued exercising control in a computer game for monetary reward. To probe this, we designed the Value of Control (VoC) task and evaluated participants’ choice behavior when presented with a series of control/no-control choice pairs whose reward point magnitudes were manipulated.
Participants first underwent the training version of the VoC task in the lab, with the goal of familiarizing them with the experimental task. Second, they completed 4 paper questionnaires given in the same order: 1) Mini mood and anxiety symptom questionnaire (Clark and Watson 1995); 2) Behavioral inhibition system/ behavioral activation system (BIS/BAS) scale (Carver and White 1994); 3) Desirability of Control Scale (Burger and Cooper 1979); 4) Internal–External Locus of Control (LOC) (Rotter 2011). Third, participants performed the testing version of the VoC task in the fMRI scanner. All computerized tasks were coded and presented using MATLAB 2015a, The MathWorks, Inc., Natick, MA, USA and Psychtoolbox 3 (Brainard 1997). Next, we describe the VoC task in more detail, including the different experimental conditions, and highlight distinctions between the training and testing phases.
The Value of Control Task
The VoC task (Fig. 1) was designed to measure an individual’s subjective value attributed to exerting control. Each trial of the VoC Task was divided into 2 parts: “Choice” and “Game” phases. The key phase-of-interest was the Choice phase, which captured a decision between exerting control (SELF-option) or relinquishing control to a computer (COMP-option) in the subsequent Game phase. During the Game phase, a card game for monetary rewards was executed by either the participant or the computer. Participants played multiple trials where they either chose between SELF- or COMP-options (Experimental condition: Mixed) or options that only varied in terms of EV (Experimental condition: Baseline). Each component of the VoC task is described next in more detail.
Figure 1.
Value of Control task. Each trial of the VoC task consisted of the Choice and Game phases. In the Choice phase, participants were presented with a pair of choices that differed based on the experimental condition. In the Game phase, depending on which option was previously chosen, either the participant (SELF-option) or the computer (COMP-option) would play the card-guessing game. Each trial ended after a quasi-exponential jitter period following the Game phase with no feedback provided for the game.
Choice phase
In the Choice phase, participants were presented with a binary choice between the SELF-option conferring behavioral control over a game and the COMP-option representing the ceding of gameplay to the computer. The 2 options were counterbalanced in terms of placement on the screen. For each option, we showed the participants the experimental points (0–20 points in increments of 2) that could be earned in the event of winning the game. Effectively, we manipulated the point magnitudes of each choice pair so that participants had to consider the reward value associated with seeking or deferring control. This 2-alternative choice design permitted us to infer how participants subjectively valued control in terms of reward EV.
The Choice phase lasted 4 seconds and was followed by a jittered 1–6-s fixation period (interstimulus interval [ISI]). A decision not captured within the 4-second Choice period was registered as a lapse for that trial and marked with a 6-s fixation period displaying the phrase “No Choice Detected!” to signal the end of that trial.
Game phase
The Game phase, which was adapted from Delgado et al. (2000), consisted of a card-guessing game where participants were shown an unknown card hiding a number ranging from 1 to 9. The objective of the game was to guess whether the hidden number was higher or lower than the number 5 (which was omitted from the deck). Depending on how participants chose in the preceding Choice phase, they could either make the guess themselves (i.e., SELF-option chosen) or the computer would make the guess on their behalf (i.e., COMP-option chosen). Importantly, regardless of how the Choice phase was played, participants had to make a single button press during the Game phase, ensuring similar motor responses across trials.
Any correct guess made by either the participant or the computer would be rewarded with the associated points added to the participant’s point bank. Any incorrect guesses by the participant or the computer yielded no net gain or loss. Experimental winning was resolved during debriefing when the participant’s point bank was revealed and converted into monetary bonus. Each trial of the Game phase lasted for 2 s and was followed by a jittered 1 to 6-s intertrial interval (ITI) showing a fixation cross to signal the end of each trial.
Training version of the task
Participants first performed the training version of the VoC task outside the fMRI scanner in order to learn the game. This session consisted of 20 forced-choice trials where participants were asked to direct their picks towards either the SELF- or COMP-option (10 trials each). The placement of each option on the screen was counterbalanced across participants. The key distinction in this version of the task and the testing version was that during training, participants received feedback on the outcome of the card-guessing game after each trial. This allowed participants to experience outcomes resulting from both SELF- and COMP-options and to gage the rate of success in the game. Participants received feedback on the Game phase where they saw whether the preceding guess (made by the participant or the computer) was correct or incorrect. Importantly, success rates for SELF- and COMP-options were equivalent at 50% and point magnitude were matched at 10 points each. At the conclusion of this training phase, participants were probed about their understanding of the game, particularly the difference between the SELF- and COMP-choices. We did not explicitly ask participants about the contingencies for the options to avoid potential instructional bias.
Testing version of the task
After training, participants performed the testing version of the VoC task consisting of 4 runs of 22 trials lasting 220 s per run. Unlike the training version of the task, participants did not experience feedback on the card-guessing game following each Game phase. In other words, while doing the task in the scanner, participants were never informed of the outcome of any guesses made by either the participant or the computer. Instead, participants’ performance and point totals were revealed to them during the debriefing session at the conclusion of the experiment. This was done to minimize the opportunity to learn and to prevent potential feedback bias on ensuing trials. In all trials following gameplay by either the participant or the computer, an ITI ensued directly after the Game phase and the trial would start again with the Choice phase. Participants were also not shown whether the computer picked higher or lower in the Game phase on trials where the COMP-option was chosen.
There were 2 experimental conditions: mixed and baseline (i.e., “controllable” and “uncontrollable”; Fig. 1). Specifically, runs 1 and 3 were mixed condition trials whereas runs 2 and 4 were a balanced combination of controllable and uncontrollable baseline trials. This run order was consistent across all participants. The 2 conditions differed only in the types of binary choices presented to the participant during the Choice phase.
Mixed condition
In mixed condition trials, the participant was presented with a choice between SELF- and COMP-options. The SELF-option was fixed at 10 points on all trials whereas the COMP-option had a balanced distribution of 0–20 points in intervals of 2 points (an additional behavioral experiment where the COMP-option was fixed at 10 points while the SELF-option varied between 0 and 20 points yields similar results and is included in the Supplementary material). This manipulation resulted in the COMP-option having a larger reward magnitude than the SELF-option in half the trials and a smaller reward magnitude in the remaining half of the trials. If participants chose the SELF-option, they were instructed to play the card-guessing game and take a gamble between 2 buttons: 1 signaling that the card number would be higher than 5 and the other 1 signaling lower than 5. In contrast, if participants chose the COMP-option, they were asked to defer gameplay to the computer and instead press a designated button to move onto the next trial. It is important to note that gameplay occurs regardless of whether SELF- or COMP-option was chosen; but the only difference is who (i.e., participant or computer) had behavioral control over the gameplay.
Baseline condition: (controllable and uncontrollable trial types)
The controllable and uncontrollable trial types collectively served as the baseline condition for the experiment. In contrast to the mixed condition, the 2 baseline trial types each featured only one type of choice (either all SELF or all COMP). For example, during the controllable trials, the participant was shown a series of choice pairs featuring 2 SELF-options. On the other hand, the uncontrollable trials gave participants a series of choice pairs with 2 COMP-options. In effect, the controllable and uncontrollable trials each encompassed sets of choice pair that differed only in its associated point magnitude but not along the dimension of controllability. It is important to note that the point magnitudes for the choice pairs in the baseline condition were matched to those in the mixed condition.
The baseline condition (i.e., controllable and uncontrollable trial types) served 2 purposes. First, these trials provided us with a behavioral measure of whether the participant understood the task and was paying attention to the information presented during the Choice phase (i.e., option type and point magnitude). Since each pair of options only differed in its point magnitude, the participant should pick the option with the higher point magnitude. Second, these trials served as a reference to which we could compare the choice pattern in the mixed condition. In the baseline condition, the participant made choices along the dimension of EV; in contrast, in the mixed condition, the participant chose along both the dimensions of EV and controllability. By comparing the choice patterns across the conditions, we can infer any difference driven by the influence of controllability in the decisions.
Neuroimaging Data Acquisition
Images were collected using a 3 T Siemens MAGNETOM Trio scanner with the 12-channel head at the Rutgers University Brain Imaging Center (RUBIC). High-resolution structural images encompassing the whole brain were acquired using a T1-weighted magnetization-prepared rapid gradient echo (MPRAGE) sequence (repetition time [TR]: 1900 ms; echo time [TE]: 2.52 ms; matrix 256 × 256; field of view [FOV]: 256 mm; voxel size 1.0 × 1.0 × 1.0 mm3; 176 slices; flip angle: 9°). The blood-oxygenation-level-dependent (BOLD) functional images were obtained using a single-shot T2*-weighted echo-planar imaging (EPI) sequence (TR: 2000 ms; TE: 25 ms; matrix 64 × 64; FOV: 192 mm; voxel size 3.0 × 3.0 × 3.0 mm3; 35 slices (0% gap); flip angle: 90°). In addition, B0 field maps (TR: 400 ms; TE1: 5.19 ms; TE2: 7.65 ms; matrix 64 × 64; FOV: 192 mm; voxel size 3.0 × 3.0 × 3.0 mm3; 35 slices (0% gap); flip angle: 60°) were collected prior to the functional images to correct for geometric distortion in the functional images.
FMRI Preprocessing
The neuroimaging data were preprocessed using SPM12 (http://www.fil.ion.ucl.ac.uk/spm/software/spm12; Ashburner J 2012). First, we defined the origin of each image to align with the anterior and posterior commissure plane (Ardekani and Bachman 2009). After we motion-corrected each time series to its first volume, we then performed spatial unwarping to minimize geometric distortions due to susceptibility artifacts (Andersson et al. 2001; Hutton et al. 2002). Next, we coregistered the mean functional image to the anatomical scan and normalized the anatomical using the unified segmentation model (Ashburner and Friston 2005). The normalized anatomical was subsequently used to reslice the functional data to standard stereotaxic space defined by the Montreal Neurological Institute (MNI). We applied a spatial smoothing at full-width half-maximum of 6 mm to the normalized functional data.
To minimize the impact of head motion on the neuroimaging data, we applied additional preprocessing steps using tools from FSL (FMRIB Software Library version 5.0.4; http://www.fmrib.ox.ac.uk/fsl; Smith et al. 2004). We detected motion spikes using the FSL tools fsl_motion_outliers. The motion spikes were evaluated with 2 metrics: 1) root-mean-square (RMS) intensity difference of each volume relative to the reference volume obtained from the first time point; and 2) frame-wise displacements calculated as the mean RMS change in rotation/translation parameters relative to the same reference volume. We subjected the metric values within a run to a boxplot threshold (75th percentile plus 1.5 times the interquartile range) and labeled volumes as spikes, which were subsequently removed via regression (Satterthwaite et al. 2013; Power et al. 2015). Across all participants, this method removed 5.8% of volumes (range: 1.0 to 11.4%). After the removal of motion spikes, no participants exhibited extreme average volume-to-volume head motion (M = 0.06 mm; range: 0.03–0.14 mm) or maximum volume-to-volume head motion (M = 0.12 mm; range: 0.05–0.31 mm). Following the removal of motion spikes, we extracted brain material from the functional images (Smith 2002) and normalized the entire 4D dataset using a single scaling factor (grand-mean intensity scaling). We also passed the images through the SUSAN (Smallest Univalue Segment Assimilating Nucleus) noise reduction filter, part of the FSL software package, using a 2 mm kernel (Smith and Brady 1997). This step allowed us to achieve greater signal-to-noise ratio while preserving the image structure. Lastly, we applied a high-pass temporal filter with a 100-s cutoff (Gaussian-weighted least-squares straight line fitting, with sigma = 50 s) to remove low frequency drift in the MR signal. Applying the temporal filter after the removal of motion spikes helps to minimize ringing artifacts (Weissenbacher et al. 2009; Carp 2013; Satterthwaite et al. 2013).
Data Analysis
Behavioral Analyses of Choices in the VoC Task
We were interested in participants’ choice behavior during the Choice phase when they were asked to pick between each choice pair. We first looked at whether participants showed any bias towards 1 of the 2 choices in each condition (i.e., mixed and baseline). For both conditions, we manipulated the reward magnitude of the choice pairs where across all trials, the 2 options had evenly matched EV, resulting in a hypothesized choice proportion of 0.5 for each option (i.e., they would pick each option 50% of the time). Therefore, within each condition, we compared participants’ choice proportions for the 2 options using a one-sample t-test against the hypothesized mean of 0.5 to investigate whether they showed a significant bias towards one of the 2 options. We used a paired t-test to test whether participants’ choice behavior differed in the 2 baseline trial types (i.e., controllable and uncontrollable).
Next, we used the participants trial-by-trial data in each condition to fit their choice behavior onto a logistic regression. By doing so, we would be able to derive at which choice pairing in mixed condition was the participant equally likely to choose either SELF- or COMP-option. The derivation of this point of equivalence (POE) provides an experimental measure of the subjective value that participants attributed to exerting control. Further details on the derivation of this POE is described in the following section. Upon deriving this POE value, we used t-tests to compare the POE for the mixed and baseline conditions to the hypothesized mean of 0 and a paired t-test to compare POE in the mixed and baseline conditions.
Finally, we examined participants’ RT during the Choice phase by running a 3 × 2 ANOVA looking at the interaction between the effect of trial types (mixed, controllable, uncontrollable) and run sequence (first vs. second run). The RT analysis allowed us to rule out differences in decisional uncertainty as a potential explanation for any choice pattern variations.
Derivation of the subjective value of control
To compare the 2 options, we first computed the EV for both options as follows:
where P is the objective success probability and V is the point magnitude rewarded. Probability (P) was deterministically set at 0.5 for both options based on the training phase feedback. The V for the COMP-option ranged from 0 to 20 points in increments of 2 while the V for the SELF-option was fixed at 10 points.
To probe participants’ choices, we fitted their trial-by-trial data onto a logistic regression. Each choice pair presented during the Choice phase was coded by the EV difference between the 2 options and this difference (i.e., minus ) served as the independent variable in our analysis. Using this EV difference and employing maximum likelihood estimation, we fitted the trial-by-trial choice data of each participant to a single logistic function of the form (Reed and Berkson 1929; Berkson 1944; Press and Wilson 1978; Davidson and MacKinnon 2004).
where, PSELF is the probability that the participant chose the SELF-option, EVCOMP and EVSELF were the EV of the COMP- and SELF-options, respectively, and γ is the slope of the logistic function (i.e., which was negative in this case), or equivalently the noise parameter.
Once data has been logistically regressed, we were interested in identifying the EV pairing where participants showed a behavioral indifference between SELF- and COMP-options. This point of indifference, or POE, would shed light on participants’ subjective valuation of the 2 options. To derive this POE for each individual participant, we analyzed each participant’s regressed behavioral data while setting the participant’s PSELF to 0.5 using the inverse of the logistic function
where PSELF is the probability of a SELF-choice, β0 is the coefficient of the constant term, and β1 is the coefficient of the predictor or independent variable. The term x represents POE—the difference in value between the 2 options (EVCOMP−EVSELF) for each participant where the participant was equally likely (i.e., PSELF = 0.5) to choose either option.
It is important to note that at the POE, and are not necessarily equivalent in terms of their EV but they are equated based on participants’ choices. Therefore, this translated into a subjective value for the SELF-option
that took into account both the , which was the objective EV of the SELF-option, and the POE, which was the intrinsic value for control.
Neuroimaging Analyses of Value of Control
Neuroimaging analyses were carried out with FSL FEAT (FMRI Expert Analysis Tool) Version 6.0 (Smith et al. 2004). All of the general linear models (GLM) described below included a regressor of no-interest for the Game phase with the duration set to 2 s and an intensity of one. In addition, all models also included a nuisance regressor for any lapse trial with the duration set to 10 s and an intensity of one. Note that all linear regressors will have an intensity set to one. All task regressors-of-interest in the GLMs were convolved with the canonical hemodynamic response function and incorporated temporal derivatives and temporal filtering.
For each participant, the data were combined across 2 runs in the second-level analysis utilizing a fixed-effects model. At the group-level analysis, we performed a mixed-effects one-sample t-tests using FEAT’s FLAME 1 + 2, which first fits the model using Bayesian modeling for mixed-effects variance estimation before processing all voxels that were close to threshold using the Metropolis–Hastings Markov Chain Monte Carlo sampling to obtain a more precise estimation of the mixed-effect variance (Woolrich et al. 2004). Unless stated otherwise, for all z-statistics images discussed, we thresholded and corrected for multiple comparisons across the whole brain using a false-discovery rate-corrected voxel-extent threshold of P < 0.05 (Worsley 2001; Lieberman and Cunningham 2009). We used MRIcroN and MRIcroGL to create the statistical overlay images (https://www.mccauslandcenter.sc.edu/crnl/tools; Rorden et al. 2007). We had specific hypotheses for each planned contrast that are described in more details in the following sections. All other findings were exploratory and are reported in the supplementary material under “Activation tables for all contrasts” (see Supplementary material).
Controllable and uncontrollable baseline trial types
In the controllable (2 SELF-option) and uncontrollable (2 COMP-option) trials, participants were asked to choose between 2 options that differed only in their reward magnitudes but not along the dimension of controllability. Therefore, we conducted a conjunction analysis on the controllable and uncontrollable trials to analyze regions associated with reward magnitude recruited by both trial types while controlling for the interaction effect between the trial types (Price and Friston 1997). We hypothesized that this analysis would yield canonical value regions such as the orbitofrontal cortex (OFC; Padoa-Schioppa and Assad 2006; Rangel et al. 2008; Schoenbaum et al. 2011; Saez et al. 2017), vmPFC (Knutson et al. 2005; Grabenhorst and Rolls 2011; Wang et al. 2016), striatum (Hare et al. 2008; Jocham et al. 2011; Barkley-Levenson and Galván 2014; Strait et al. 2015), and anterior cingulate cortex (ACC; Kennerley et al. 2011; Rushworth et al. 2012; Kolling et al. 2016; Shenhav et al. 2016; Hyman et al. 2017). In particular, prior studies that have implicated these regions (i.e., OFC, vmPFC, striatum, and ACC) in encoding the magnitude associated with potential reward have done so using both human fMRI work (Knutson et al. 2005; Diekhof et al. 2012) and animal electrophysiological recordings (Padoa-Schioppa and Assad 2006; Hamid et al. 2015).
To carry out the conjunction analysis, we performed a parametric GLM and created participant-specific design matrices containing the following task regressors: 1) a parametric regressor encoding controllable (SELF) choices with the duration corresponding to the duration of the Choice phase and the parametric modulation set to the higher EV of each choice pair; 2) a parametric regressor encoding uncontrollable (COMP) choices with the duration corresponding to the duration of the Choice phase and the parametric modulation set to the higher EV of each choice pair. This model also included a regressor of no-interest for the Game phase with the duration set to 2 s and an intensity of one, and a nuisance regressor for any lapse trial with the duration set to 10 s and an intensity of one. To obtain conjunction activation, we masked regressor 1) with regressor 2).
In addition to the conjunction analysis, we did a second analysis by contrasting the controllable and the uncontrollable trials to probe neural systems involved in encoding the opportunity for control during gameplay. Based on previous studies from our lab showing that cues associated with control (i.e., having choices) in contrast to cues associated with no control (i.e., no choices) recruited reward-processing regions such as the striatum (Leotti and Delgado 2011, 2014), we hypothesized that the contrast of controllable–uncontrollable trials would reveal activation in the striatum and that this predicted activation would be related to participants’ inherent preference for control as measured by their LOC score. In addition, our hypothesis on striatal activation was also drawn from previous experiments showing that the presence of controllability in the external environment was associated with dopamine release into the NAcc (Cabib and Puglisi-Allegra 2012; Cockburn et al. 2014; Ikemoto et al. 2015).
For the second analysis, we built a GLM by creating participant-specific design matrices containing a linear regressor encoding controllable (all SELF) choices with the duration corresponding to the duration of the Choice phase and the intensity set to one as well as a linear regressor encoding uncontrollable (all COMP) choices with the duration corresponding to the duration of the Choice phase and the intensity set to one. This model also included a regressor of no-interest for the Game phase with the duration set to 2 s and an intensity of one, and a nuisance regressor for any lapse trial with the duration set to 10 s and an intensity of one. Our group-level contrasts included controllable minus uncontrollable choices and vise versa.
Mixed condition
We reasoned that in the mixed trials, participants were choosing between each choice pair by assigning a subjective value to the SELF-option. This subjective value had to encompass both EV computation and the subjective valuation of control. We effectively isolated this subjective valuation of control in our POE measure (for additional details, see section on “Derivation of the Subjective Value of Control”). By leveraging this POE measure, we could examine whether the subjective value of control was encoded by neural regions associated with the computation of affective value such as the vmPFC (Delgado 2007; Haber and Knutson 2010; Rushworth et al., 2012; Bartra et al. 2013; Delgado et al. 2016). We had a particular hypothesis on the vmPFC as a potential region for encoding the POE measure for 2 reasons. First, prior studies collectively suggested that the vmPFC could serve as the region responsible for representing the subjective values associated with choices across different types of reward (Levy and Glimcher 2012). Second, the vmPFC has been suggested to be necessary for the behavioral bias that animals show towards detecting and exercising control (Amat et al. 2005; Maier et al. 2006; Maier and Watkins 2010).
We performed a GLM analysis with participant-specific design matrices containing the following regressors for the mixed condition: 1) a linear regressor encoding the SELF-choices with the duration corresponding to the duration of the Choice phase and an intensity of 1; and 2) a linear regressor encoding the COMP-choices with the duration corresponding to the duration of the Choice phase and an intensity of 1. This model also included a regressor of no-interest for the Game phase with the duration set to 2 s and an intensity of 1, and a nuisance regressor for any lapse trial with the duration set to 10 s and an intensity of 1. At the group-level analysis, we added the participant-specific POE into the GLM as a covariate and performed a mixed-effects one-sample t-tests on the contrast between SELF-choices and COMP-choices (i.e., SELF-choices–COMP-choices).
Results
Behavioral Results
The analysis focused on participants’ behavior in the Choice phase of the VoC task because how they picked between the binary options would inform on how much perceived control contributed to decision making. Therefore, we probed participants choice pattern by first examining whether they showed any bias towards one of the 2 options. In the mixed condition, participants showed a preference for the SELF-option by choosing it 57.1% of the time (Fig. 2A; t[26] = 3.55, P = 0.0015). In contrast, participants showed no bias towards either option in each choice pair in the controllable and uncontrollable trial types (i.e., they chose COMP1 51% of the time in uncontrollable trials) (t[26] = 0.73, P = 0.47) and SELF1 51% in controllable trials (Fig. 2B; t[26] = 1.00, P = 0.32). Because the point magnitude for each choice pair in the mixed and baseline conditions were matched, the bias shown for the SELF-option in the mixed condition suggested that participants subjectively inflated the value of said option over its EV.
Figure 2.
Behavioral findings. We compared participants’ choice proportion for 1 of the 2 options (i.e., SELF in mixed, SELF1 in controllable, COMP1 in uncontrollable) against the hypothesized mean of 0.5. (A) In the mixed condition, participants showed a significant bias towards the SELF-option. (B) In contrast, in the 2 baseline trial types, participants did not show a significant bias towards either option in each choice pair. Note that SELF1 for the controllable trials indicated 1 of the 2 SELF-options presented to participants whereas COMP1 indicated one of the 2 COMP options in the uncontrollable trials. In addition, we found no significant difference in the choice bias and pattern between the 2 baseline trial types. (C) Regression analysis conducted on participants’ choice patterns revealed that the POE for the mixed condition was significantly greater than 0 (POE = 3.06) in contrast to the POE of 0.16 for the baseline condition. The x-axis indicated the reward expected value difference between each choice pair such that in the mixed condition, x-axis less than 0 indicated a larger SELF EV compared with COMP EV and vice versa for x-axis greater than 0. The y-axis indicated the proportion of choices which for the mixed condition would be proportion of SELF-choices and for the baseline condition would be proportion of fixed choices. The horizontal line indicated a choice proportion of 0.5 and intersections with the curved lines represent the POE for each condition.
For the baseline condition, participants picked the option carrying the higher EV 88% (SD: 4.6) of the time in the controllable trials and 87% (SD: 4.8) of the time in the uncontrollable trials, suggesting that they overwhelmingly deferred to the choice with the higher EV in the baseline condition. Given that there was no statistical difference in participants’ choice pattern between the controllable and uncontrollable trials (t[10] = 0.19, P = 0.85), we combined the 2 baseline trial types in subsequent analyses.
To examine how much controllability contributed to decision making during the Choice phase, we performed a logistic regression analysis on participants’ trial-by-trial data to extract individual participant’s POE. If controllability did not contribute to decision making, participants’ POE should be 0 to indicate that participants were equally likely to choose either option when there was no EV difference between the choice pair. In other words, the behavioral equivalence derived from participants’ choice pattern was established from the reward EV of the choice pairs. Based on participants’ choice bias from the previous analysis, we predicted that the POEs for the combined baseline condition would be close to 0 whereas the POEs extracted from the mixed condition would be significantly different from 0. We tested this hypothesis using a one-sample t-test against the predicted mean of 0.
For the pooled baseline condition data (i.e., controllable and uncontrollable), the regression analysis revealed a mean participant POE of 0.16 (Fig. 2C, solid line; SD = 1, range = −2.27 to 2.98), and this was found to not be significantly different from the expected POE of 0 (t[26] = 0.83, P = 0.41), suggesting that participants chose based on EV. In contrast, for the mixed condition, the regression analysis yielded an average participant POE of 3.06 (Fig. 2C, dashed line; SD = 6.8, range = −2.02 to 33.44), with a beta value of −0.41 and odds ratio of 0.67 (z = −17.82, P < 0.001). This mean POE in the mixed condition was significantly different from the expected POE of 0 (t[26] = 2.33, P = 0.028), suggesting that EV was not the only factor influencing the choices. Comparing the mixed and baseline conditions, we found that the POEs across participants were significantly different (t[26] = 2.16, P = 0.04).
Taken together, the extracted POEs for the mixed condition could be interpreted as the SELF-option carrying an average of 30% increase in value compared with the COMP-option, suggesting that participants placed a higher subjective value on the SELF-option. This 30% increase for the SELF-option was derived from the mean POE measure (POE = 3.06) where a 10-point SELF-option was found to be behaviorally equivalent to a 13-point COMP-option. This increase in the value of the SELF-option was only observed when participants were asked to choose between a SELF- and a COMP-option but not when 2 SELF-options (i.e., controllable trial type) were presented to participants. Collectively, our behavioral analyses revealed that in the mixed condition, participants were making their decisions based on both reward magnitude and the presence of controllability over gameplay. Specifically, exactly how much controllability contributed in terms of reward value to the decision was effectively captured by POE measure.
Reaction Time
We also quantified participants’ reaction time (RT) during the Choice phase across trial types (mixed: M = 1.13, SD = 0.23; Controllable: M = 1.1; SD = 0.16; Uncontrollable: M = 1.08; SD = 0.14). We found that participants’ RT did not differ across trial types (F[2156] = 0.78, P = 0.4580) and run sequence (F[1156] = 0.73, P = 0.3930). We also did not find a significant interaction between trial type and run sequence (F[2156] = 0.04, P = 0.9608). Similarly, participants did not differ significantly in their SELF- and COMP-choice RTs during the controllable and uncontrollable trials, respectively (t[26] = −1.44, P = 0.16). Reaction time between the SELF- and COMP-choices in the mixed condition was marginally significant (t[26] = 1.71 P = 0.099), with slower RTs for the SELF-choices.
Neuroimaging Analyses
As detailed in the behavioral results section, participants’ choice behavior demonstrated that, in the controllable and uncontrollable trials, they overwhelmingly picked the option with the higher reward magnitude, suggesting that their choices were driven by the reward EV. Therefore, we hypothesized that our parametric analysis of these 2 trial types should yield activation in canonical value-encoding regions such as the OFC, ACC, striatum, and vmPFC (Rangel and Hare 2010; Bartra et al. 2013). After correcting for whole-brain multiple comparisons, the conjunction analysis revealed activation in the ventral striatum (peak z-stats = 3.6 at MNIx, y, z = −20, 16, −4, PFDR voxel-corrected < 0.01, 71 voxels), ACC (peak z-stats = 5.5 at MNIx, y, z = 3, 11, 43, PFDR voxel-corrected < 0.01, 2041 voxels) and OFC (peak z-stats = 3.5 at MNIx, y, z = 37, 22, −12, PFDR voxel-corrected < 0.01, 38 voxels).
Neural Correlates Underlying the Opportunity for Control
Although participants’ choice behaviors were similar in the 2 baseline trial types, we argued that the 2 trial types were different on the basis that participants made only SELF-choices in the controllable trials and only COMP-choices in the uncontrollable trials. Therefore, we directly contrasted the controllable and uncontrollable responses across the 2 trial types to identify regions whose activation changed according to the presence of control, or more aptly, the opportunity to exert control. Our whole-brain analysis identified the ventral striatum, particularly the NAcc (Fig. 3A; peak z-stats = 3.9 at MNIx, y, z = −6, 6, −8, PFDR voxel-corrected < 0.05, 16 voxels) and anterior midcingulate cortex (aMCC; Fig. 3A; peak z-stats = 4.2 at MNIx, y, z = −2, 10, 43, PFDR voxel-corrected < 0.01, 127 voxels) exhibiting greater responses to the controllable SELF-choices relative to the uncontrollable COMP-choices but not the reverse contrast.
Figure 3.
Neural correlates for value of perceived control. (A) To identify brain regions that were recruited in the controllable trials, we conducted a parametric model by contrasting the 2 baseline trial types (controllable–uncontrollable). After correcting for whole-brain voxel-based multiple comparisons, we found that the NAcc and ACC showed stronger activation during the controllable trials relative to the uncontrollable trials. (B) A negative correlation between NAcc activity and each participant’s LOC score was observed, with higher striatal activation corresponding to more internal LOC and greater subjective preference for control. (C) To identify brain regions whose activation tracked increasing subjective value of control represented by participants’ POE measure, we performed a GLM of the mixed condition and added a subject-level POE covariate to parametrically modulate the contrast of SELF-choices–COMP-choices. After correcting for multiple comparisons across the whole brain, we found that the vmPFC responded to increasing POE measure.
Based on our a priori hypothesis regarding the ventral striatum, we tested whether the activity in this region could be related to participants’ inherent preference for control by conducting a post hoc analysis comparing the striatal responses to the participants’ LOC scores (M = 5.82, SD = 2.00, normally distributed using skewness and kurtosis test for normality [P = 0.44]) obtained using a questionnaire at the start of the experiment (all other questionnaire results are reported in the Supplementary Material under section titled “Questionnaire results”). In particular, the Internal–External LOC scale has been a longstanding subjective scale to measure individual differences in how people generally perceive both the presence and the significance of having control in their lives (Lefcourt 2014). The LOC concept centers on the differences in perception of control across individuals where someone with a more internal locus of control is more likely to have stronger beliefs for perceiving control in his or her life that is captured in a lower LOC score (Rotter 2011). Using an anatomical striatal mask, we extracted each participant’s contrast of parameter estimate for the striatal activation and found that this measure correlated negatively with participants’ LOC scores (Fig. 3B; R = −0.392, P = 0.043). This suggested that participants with stronger striatal activation in response to the opportunity for control have a more internal locus of control (i.e., they believed in themselves and preferred more control) represented by a lower LOC score (Rotter 1966, 2011).
Neural Correlates of Subjective Value of SELF-Choices
From the regression analysis, we showed that in the mixed condition participants showed a clear bias towards the option conferring control (i.e., SELF-option). This led to the derivation of the POE measure, which was the experimental measure for the subjective value of control. Using this measure as a parametric covariate added to our GLM, we tested for regions that tracked this POE measure when participants selectively chose the SELF-option over the COMP-option. We found that in the contrast of SELF-choices minus COMP-choices, the parametric modulation of the POE covariate yielded activation in the vmPFC cortex (Fig. 3C; peak z-stats = 3.8 at MNIx, y, z = −6, 32, −14, PFDR voxel-corrected < 0.05, 12 voxels), potentially suggesting that a higher subjective value of control, as captured by the POE measure, is encoded in participants’ vmPFC BOLD signals.
Discussion
In this study, we examined the neural basis of subjective value of perceived control and how it impacts decision making. We found that perceiving control over a potential reward resulted in participants inflating the value of the associated outcome by 30%. This value inflation was sufficient to make the option conferring control desirable even at a cost to participants, which was consistent with previous experiments showing that the general partiality towards control translated into a “control premium” (Owens et al. 2014; Bobadilla-Suarez et al. 2017). Importantly, we were able to extend these findings by quantifying the subjective value of perceived control embedded within reward-seeking behaviors highlighting that control bears desirable qualities. Critically, the vmPFC computed and tracked this subjective value of control within the reward-seeking decision.
There were 2 baseline trial types (i.e., controllable and uncontrollable) that served as our experimental reference for the behavioral analyses in the condition of interest (i.e., mixed). We leveraged the differences in the 2 baseline trial types where control was always presented in one (i.e., controllable trials) and always absent in the other (i.e., uncontrollable trials) to find that the ventral striatum (i.e., NAcc) and the aMCC were engaged when there was an opportunity for control in the controllable trials. In line with previous research concluding that perceived control may have inherent affective properties that makes it subjectively desirable (for review, see Leotti et al. 2010), our current experiment strengthened this argument by presenting evidence that the opportunity for control in the environment recruited key reward-processing regions such as the ventral striatum (Apicella et al. 1991; Delgado 2007; Wang et al. 2016). We also observed activation in the aMCC in response to the controllable compared with the uncontrollable trials. This observation is consistent with a previous experiment where aMCC is more engaged during free compared with forced motor choice (Hoffstaedter et al. 2012) and with animal studies showing that neurons in the aMCC respond to anticipated reward-related motor behaviors (Shima and Tanji 1998; Akkal et al. 2002).
The striatal activation in response to the opportunity for control was tied to how much the participants subjectively preferred exercising control as a function of their LOC score. The LOC scale has previously been applied in experimental settings to demonstrate that those with a more internal locus were oriented towards behaviors and activities meeting their higher expectancy of control (Joe 1971; Dembroski et al. 1984; Hashimoto and Fukuhara 2004). Extending from these findings, a participant with a lower LOC score (i.e., more internal locus of control) would be predicted to have a greater inclination for having control (Rotter 2011) and as such, we observed that this translated into stronger striatal activity when control was present in the controllable trials. While this observation is associated with a typical neuroimaging sample size (N = 27), it is important to replicate this individual difference effect in future studies. Taken together, the striatum was recruited when there was an opportunity for control in the environment and the strength of its activation was related to the individual’s inherent preference for control.
Turning to the mixed condition and our experimental measure of the subjective value of control (i.e., POE), we found that the vmPFC served as the neural correlate subserving the computation of how much perceived control influenced decision making. That is, vmPFC was recruited to encode a higher subjective value of control as captured by the POE measure. This suggests that beyond the vmPFC’s involvement when an organism perceives and chooses to exercise control in the environment (Amat et al. 2005; Maier et al. 2006; Christianson et al. 2009), the vmPFC may also have a more fine-tuned role to engage in computing how much the organism actually desired control. Accordingly, individuals who showed greater behavioral bias towards seeking control (i.e., stronger desire for control) also had a higher subjective value of control that was tracked by greater vmPFC activation. The vmPFC has been implicated in encoding a “common currency” for the valuation of choices made between different rewards (for review, see Knutson et al. 2005; Levy and Glimcher 2012; Rushworth et al., 2012; Bartra et al. 2013), and the observation that vmPFC tracks the subjective value of control lends support to the idea that perceiving and exercising control has positive affective properties to make it valuable.
Taken together with the observation that the striatum was involved in encoding the opportunity for control, we argue that participants’ inherent behavioral bias towards seeking and retaining control was sustained by the rewarding and motivating nature of perceived control. Our current finding expands upon prior animal studies (Amat et al. 2005; Maier and Watkins 2010) to suggest that the role of vmPFC in subserving control was contingent on how much positive value the organism attributed to seeking control in the decision-making process. The perception of control is not a binary on/off switch but rather, its contribution to adaptive behaviors is dependent on how it is subjectively valued. This graded-value feature of control allows for the possibility of circumstances where control is voluntarily relinquished (Sunstein 2017) or even not desired (Iyengar and Lepper 2000; Schwartz 2004). We postulate that one of the driving forces potentially subserving this inherent preference for control is the value of information (Bordia et al. 2004; Tricomi and Fiez 2012), even if useless (Eliaz and Schotter 2010), which can lower uncertainty (Behrens et al. 2007). By having agency over the gameplay, participants could subjectively interpret that they have more information on the game and hence contribute to their bias towards the SELF-option.
In the mixed condition of our task, participants had to evaluate both the reward value and the anticipated effort cost associated with choosing either option before they reach an optimizing decision to retain or forgo control. According to the EV of control theory (Shenhav et al. 2013), the higher-level integration of control-related reward and cost computation is subserved by the dorsal anterior cingulate cortex (dACC) while lower-level direct representation of affective value of control is associated with regions such as ventral PFC and striatum. As such, our current findings aligned with this framework by showing that the ventral striatum encoded the affective signal associated with presence of control in the baseline controllable trials and that vmPFC tracked the value of control exertion in the mixed condition. It would be worthwhile for future research to also manipulate task demands (e.g., increasing or decreasing task difficulty) so as to not only replicate the striatum and vmPFC observation of encoding reward-related value signals associated with perceived control, but also the involvement of dACC in higher-level integration of reward and cost signals for action selection.
Another potential interpretation of our findings is that participants’ SELF-option choices were driven by the belief in the probability of success for a particular option. However, we note that all participants in our experiment first underwent a training version of the task where they experienced feedback for both options that was deterministically set at 50% correct. Therefore, they started the game under the belief that both SELF- and COMP-options could result in successful and unsuccessful outcomes. In addition, in a prior study by our group, it was found that the opportunity for choice as a proxy for perceived control elicited different subjective ratings and neural activations compared with an option that conferred a belief of higher success probability (Leotti and Delgado 2011). Nevertheless, in our current paradigm, because we only manipulated reward EV via varying the reward magnitude, future studies should separately vary the probability and magnitude component of reward EV so as to tease apart whether each component exert differential influence on the subjective value of control. In a similar vein, given the possibility that the value of reward can potentially interact with the subjective value of control, future studies should investigate any contextual effects related to varying the size of the overall reward at stake.
Yet another potential interpretation of participants’ choice bias is that they picked the control-conferring option in order to stay engaged in the scanner. However, a motor response was required for both options in the choice phase and their subsequent game phase and reaction time data was not different across the conditions. Further, participants’ postexperimental debriefing suggested that they were engaged in the task while in the scanner. Accordingly, we argue that the bias that participants showed towards the control-conferring option was most likely driven by their inherent preference for perceiving and exercising control. This notion of staying engaged in the scanner begets the possibility that arousal represents another potential driver of SELF-choices during the task. Control can be perceived as both rewarding and inherently desired, and that perceiving control has been tied to both increased (Ramsey and Etcheverry 2013) and decreased arousal (Gallagher et al. 2014). Thus, an interesting future direction may be to more directly assess arousal via measures such as skin conductance or pupil size in order to gain insights into how arousal influences participants’ choice behavior in the VoC task.
In conclusion, we found that participants showed a clear preference towards exerting control that was captured as the subjective value of control embedded in the reward EV. This behavioral bias was subserved by the ventral striatum mediating the opportunity for control in the environment and the vmPFC tracking this subjective value of control (i.e., POE). These findings collectively suggest that the computation of the value of perceived control in decision making is rooted within corticostriatal circuitry typically associated with reward-related processing and valuation. This is important to consider given the prevalence of the loss of control in many psychopathologies such as addiction, post-traumatic stress disorder and depression (Glass and McKnight 1996; Frazier et al. 2004; Bechara 2005). Indeed, the perceived loss of control is a hallmark of disorders like addiction where loss of behavioral control to resist the addicted substance are observed (for review, see Everitt and Robbins 2016; Koob and Volkow 2016). Ultimately, measuring the subjective valuation of control and understanding its source can help to both reconcile changes reported in diseased states and also to inform us on questions regarding the inherent preference for control. The gained knowledge of the relationship between perceived control and adaptive behavior can foster development of better treatment plans and methods to predict susceptibility to psychopathologies.
Supplementary Material
Notes
We thank John McClure for helpful comments on previous drafts of the article. Conflict of Interest: The authors declare no competing financial interests.
Funding
This study was supported by funding from the National Institutes of Health to M.R.D. (DA027764).
References
- Abramson LY, Seligman ME, Teasdale JD. 1978. Learned helplessness in humans: critique and reformulation. J Abnorm Psychol. 87:49. [PubMed] [Google Scholar]
- Ajzen I. 1991. The theory of planned behavior. Organ Behav Hum Decis Process. 50:179–211. [Google Scholar]
- Akkal D, Bioulac B, Audin J, Burbaud P. 2002. Comparison of neuronal activity in the rostral supplementary and cingulate motor areas during a task with cognitive and motor demands. Eur J Neurosci. 15:887–904. [DOI] [PubMed] [Google Scholar]
- Amat J, Baratta MV, Paul E, Bland ST, Watkins LR, Maier SF. 2005. Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat Neurosci. 8:365–371. [DOI] [PubMed] [Google Scholar]
- Andersson JL, Hutton C, Ashburner J, Turner R, Friston K. 2001. Modeling geometric deformations in EPI time series. Neuroimage. 13:903–919. [DOI] [PubMed] [Google Scholar]
- Apicella P, Ljungberg T, Scarnati E, Schultz W. 1991. Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res. 85:491–500. [DOI] [PubMed] [Google Scholar]
- Ardekani BA, Bachman AH. 2009. Model-based automatic detection of the anterior and posterior commissures on MRI scans. Neuroimage. 46:677–682. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ashburner J. 2012. SPM: a history. Neuroimage. 62:791–800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ashburner J, Friston KJ. 2005. Unified segmentation. Neuroimage. 26:839–851. [DOI] [PubMed] [Google Scholar]
- Bandura A. 1977. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 84:191. [DOI] [PubMed] [Google Scholar]
- Barkley-Levenson E, Galván A. 2014. Neural representation of expected value in the adolescent brain. Proc Natl Acad Sci USA. 111:1646–1651. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bartra O, McGuire JT, Kable JW. 2013. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage. 76:412–427. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bechara A. 2005. Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective. Nat Neurosci. 8:1458–1463. [DOI] [PubMed] [Google Scholar]
- Behrens TE, Woolrich MW, Walton ME, Rushworth MF. 2007. Learning the value of information in an uncertain world. Nat Neurosci. 10:1214–1221. [DOI] [PubMed] [Google Scholar]
- Berkson J. 1944. Application of the logistic function to bio-assay. J Am Stat Assoc. 39:357–365. [Google Scholar]
- Bobadilla-Suarez S, Sunstein CR, Sharot T. 2017. The intrinsic value of choice: the propensity to under-delegate in the face of potential gains and losses. J Risk Uncertain. 54:187–202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bordia P, Hunt E, Paulsen N, Tourish D, DiFonzo N. 2004. Uncertainty during organizational change: Is it all about control? Eur J Work Organ Psychol. 13:345–365. [Google Scholar]
- Bown NJ, Read D, Summers B. 2003. The lure of choice. J Behav Decis Making. 16:297. [Google Scholar]
- Brainard DH. 1997. The Psychophysics Toolbox. Spat Vis. 10:433–436. [PubMed] [Google Scholar]
- Burger JM, Cooper HM. 1979. The desirability of control. Motiv Emot. 3:381–393. [Google Scholar]
- Cabib S, Puglisi-Allegra S. 2012. The mesoaccumbens dopamine in coping with stress. Neurosci Biobehav Rev. 36:79–89. [DOI] [PubMed] [Google Scholar]
- Carp J. 2013. Optimizing the order of operations for movement scrubbing: comment on Power et al. Neuroimage. 76:436–438. [DOI] [PubMed] [Google Scholar]
- Carver CS, White TL. 1994. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS Scales. J Pers Soc Psychol. 67:319–333. [Google Scholar]
- Catania AC, Sagvolden T. 1980. Preference for free choice over forced choice in pigeons. J Exp Anal Behav. 34:77–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Christianson JP, Thompson BM, Watkins LR, Maier SF. 2009. Medial prefrontal cortical activation modulates the impact of controllable and uncontrollable stressor exposure on a social exploration test of anxiety in the rat. Stress. 12:445–450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Clark L, Watson D. 1995. The mini mood and anxiety symptom questionnaire (Mini-MASQ). Unpublished manuscript, University of Iowa.
- Cockburn J, Collins AG, Frank MJ. 2014. A reinforcement learning mechanism responsible for the valuation of free choice. Neuron. 83:551–557. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Davidson R, MacKinnon JG. 2004. Econometric theory and methods. New York: Oxford University Press. [Google Scholar]
- Delgado MR. 2007. Reward‐related responses in the human striatum. Ann N Y Acad Sci. 1104:70–88. [DOI] [PubMed] [Google Scholar]
- Delgado M, Beer J, Fellows L, Huettel S, Platt M, Quirk G, Schiller D. 2016. Viewpoints: dialogues on the functional role of the ventromedial prefrontal cortex. Nat Neurosci. 19:1545–1552. [DOI] [PubMed] [Google Scholar]
- Delgado MR, Nystrom LE, Fissell C, Noll D, Fiez JA. 2000. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol. 84:3072–3077. [DOI] [PubMed] [Google Scholar]
- Dembroski TM, MacDougall JM, Musante L. 1984. Desirability of control versus locus of control: relationship to paralinguistics in the Type A interview. Health Psychol. 3:15. [DOI] [PubMed] [Google Scholar]
- Diekhof EK, Kaps L, Falkai P, Gruber O. 2012. The role of the human ventral striatum and the medial orbitofrontal cortex in the representation of reward magnitude—an activation likelihood estimation meta-analysis of neuroimaging studies of passive reward expectancy and outcome processing. Neuropsychologia. 50:1252–1266. [DOI] [PubMed] [Google Scholar]
- Eliaz K, Schotter A. 2010. Paying for confidence: an experimental study of the demand for non-instrumental information. Games Econ Behav. 70:304–324. [Google Scholar]
- Everitt BJ, Robbins TW. 2016. Drug addiction: updating actions to habits to compulsions ten years on. Annu Rev Psychol. 67:23–50. [DOI] [PubMed] [Google Scholar]
- Frazier P, Steward J, Mortensen H. 2004. Perceived control and adjustment to trauma: a comparison across events. J Soc Clin Psychol. 23:303. [Google Scholar]
- Fujiwara J, Usui N, Park SQ, Williams T, Iijima T, Taira M, Tsutsui K-I, Tobler PN. 2013. Value of freedom to choose encoded by the human brain. J Neurophysiol. 110:1915–1929. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gallagher MW, Bentley KH, Barlow DH. 2014. Perceived control and vulnerability to anxiety disorders: a meta-analytic review. Cognit Ther Res. 38:571–584. [Google Scholar]
- Glass D, McKnight J. 1996. Perceived control, depressive symptomatology, and professional burnout: a review of the evidence. Psychol Health. 11:23–48. [Google Scholar]
- Grabenhorst F, Rolls ET. 2011. Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci. 15:56–67. [DOI] [PubMed] [Google Scholar]
- Haber SN, Knutson B. 2010. The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology. 35:4–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, Kennedy RT, Aragona BJ, Berke JD. 2015. Mesolimbic dopamine signals the value of work. Nat Neurosci. 19:117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hare TA, O’Doherty J, Camerer CF, Schultz W, Rangel A. 2008. Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J Neurosci. 28:5623–5630. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hashimoto H, Fukuhara S. 2004. The influence of locus of control on preferences for information and decision making. Patient Educ Couns. 55:236–240. [DOI] [PubMed] [Google Scholar]
- Hoffstaedter F, Grefkes C, Zilles K, Eickhoff SB. 2012. The “what” and “when” of self-initiated movements. Cereb Cortex. 23:520–530. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hutton C, Bork A, Josephs O, Deichmann R, Ashburner J, Turner R. 2002. Image distortion correction in fMRI: a quantitative evaluation. Neuroimage. 16:217–240. [DOI] [PubMed] [Google Scholar]
- Hyman JM, Holroyd CB, Seamans JK. 2017. A novel neural prediction error found in anterior cingulate cortex ensembles. Neuron. 95:447–456. e443. [DOI] [PubMed] [Google Scholar]
- Ikemoto S, Yang C, Tan A. 2015. Basal ganglia circuit loops, dopamine and motivation: a review and enquiry. Behav Brain Res. 290:17–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Iyengar SS, Lepper MR. 2000. When choice is demotivating: can one desire too much of a good thing? J Pers Soc Psychol. 79:995–1006. [DOI] [PubMed] [Google Scholar]
- Jocham G, Klein TA, Ullsperger M. 2011. Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices. J Neurosci. 31:1606–1613. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Joe VC. 1971. Review of the internal-external control construct as a personality variable. Psychol Rep. 28:619–640. [DOI] [PubMed] [Google Scholar]
- Kennerley SW, Behrens TE, Wallis JD. 2011. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat Neurosci. 14:1581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Knutson B, Taylor J, Kaufman M, Peterson R, Glover G. 2005. Distributed neural representation of expected value. J Neurosci. 25:4806–4812. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kolling N, Wittmann MK, Behrens TE, Boorman ED, Mars RB, Rushworth MF. 2016. Value, search, persistence and model updating in anterior cingulate cortex. Nat Neurosci. 19:1280. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Koob GF, Volkow ND. 2016. Neurobiology of addiction: a neurocircuitry analysis. Lancet Psychiatry. 3:760–773. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lefcourt HM. 2014. Locus of control: current trends in theory & research. New York, NY: Psychology Press. [Google Scholar]
- Leotti LA, Delgado MR. 2011. The inherent reward of choice. Psychol Sci. 22(10):1310–1318. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leotti LA, Delgado MR. 2014. The value of exercising control over monetary gains and losses. Psychol Sci. 25:596–604. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leotti LA, Iyengar SS, Ochsner KN. 2010. Born to choose: the origins and value of the need for control. Trends Cogn Sci. 14:457–463. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Levy DJ, Glimcher PW. 2012. The root of all value: a neural common currency for choice. Curr Opin Neurobiol. 22:1027–1038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lieberman MD, Cunningham WA. 2009. Type I and Type II error concerns in fMRI research: re-balancing the scale. Soc Cogn Affect Neurosci. 4:423–428. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ly V, Wang KS, Bhanji J, Delgado MR. 2019. A Reward-Based Framework of Perceived Control. Front Neurosci. 13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Maier SF, Amal J, Baratta MV, Paul E, Watkins LR. 2006. Behavioral control, the medial prefrontal cortex, and resilience. Dialogues Clin Neurosci. 8:397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Maier SF, Seligman ME. 1976. Learned helplessness: theory and evidence. J Exper Psychol. 105:3. [Google Scholar]
- Maier SF, Seligman ME. 2016. Learned helplessness at fifty: insights from neuroscience. Psychol Rev. 123:349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Maier SF, Watkins LR. 2010. Role of the medial prefrontal cortex in coping and resilience. Brain Res. 1355:52–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Owens D, Grossman Z, Fackler R. 2014. The control premium: a preference for payoff autonomy. Am Econ J. 6:138–161. [Google Scholar]
- Padoa-Schioppa C, Assad JA. 2006. Neurons in the orbitofrontal cortex encode economic value. Nature. 441:223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Power JD, Schlaggar BL, Petersen SE. 2015. Recent progress and outstanding issues in motion correction in resting state fMRI. Neuroimage. 105:536–551. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Press SJ, Wilson S. 1978. Choosing between logistic regression and discriminant analysis. J Am Stat Assoc. 73:699–705. [Google Scholar]
- Price CJ, Friston KJ. 1997. Cognitive conjunction: a new approach to brain activation experiments. Neuroimage. 5:261–270. [DOI] [PubMed] [Google Scholar]
- Ramsey AT, Etcheverry PE. 2013. Aligning task control with desire for control: implications for performance. Basic Appl Soc Psych. 35:467–476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rangel A, Camerer C, Montague PR. 2008. A framework for studying the neurobiology of value-based decision making. Nat Rev Neurosci. 9:545–556. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rangel A, Hare T. 2010. Neural computations associated with goal-directed choice. Curr Opin Neurobiol. 20:262–270. [DOI] [PubMed] [Google Scholar]
- Reed LJ, Berkson J. 1929. The application of the logistic function to experimental data. J Phys Chem. 33:760–779. [Google Scholar]
- Rorden C, Karnath H-O, Bonilha L. 2007. Improving lesion-symptom mapping. J Cogn Neurosci. 19:1081–1088. [DOI] [PubMed] [Google Scholar]
- Rotter JB. 1966. Generalized expectancies for internal versus external control of reinforcement. Psychol Monogr. 80:1. [PubMed] [Google Scholar]
- Rotter J. 2011. Rotter Internal-External Locus of Control Scale. 28 Measures of Locus of Control. 10.
- Roy M, Shohamy D, Wager TD. 2012. Ventromedial prefrontal-subcortical systems and the generation of affective meaning. Trends Cogn Sci. 16:147–156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rushworth MF, Kolling N, Sallet J, Mars RB. 2012. Valuation and decision-making in frontal cortex: one or many serial or parallel systems? Curr Opin Neurobiol. 22:946–955. [DOI] [PubMed] [Google Scholar]
- Saez RA, Saez A, Paton JJ, Lau B, Salzman CD. 2017. Distinct roles for the amygdala and orbitofrontal cortex in representing the relative amount of expected reward. Neuron. 95:70–77.e73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Satterthwaite TD, Elliott MA, Gerraty RT, Ruparel K, Loughead J, Calkins ME, Eickhoff SB, Hakonarson H, Gur RC, Gur RE. 2013. An improved framework for confound regression and filtering for control of motion artifact in the preprocessing of resting-state functional connectivity data. Neuroimage. 64:240–256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schoenbaum G, Takahashi Y, Liu TL, McDannald MA. 2011. Does the orbitofrontal cortex signal value? Ann N Y Acad Sci. 1239:87–99. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schwartz B. 2004. The paradox of choice. New York, NY: Harper Collins. [Google Scholar]
- Shenhav A, Botvinick Matthew M, Cohen Jonathan D. 2013. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron. 79:217–240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shenhav A, Straccia MA, Botvinick MM, Cohen JD. 2016. Dorsal anterior cingulate and ventromedial prefrontal cortex have inverse roles in both foraging and economic choice. Cogn Affect Behav Neurosci. 16:1127–1139. [DOI] [PubMed] [Google Scholar]
- Shima K, Tanji J. 1998. Role for cingulate motor area cells in voluntary movement selection based on reward. Science. 282:1335–1338. [DOI] [PubMed] [Google Scholar]
- Smith SM. 2002. Fast robust automated brain extraction. Hum Brain Mapp. 17:143–155. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith SM, Brady JM. 1997. SUSAN—a new approach to low level image processing. Int J Comput Vis. 23:45–78. [Google Scholar]
- Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TE, Johansen-Berg H, Bannister PR, De Luca M, Drobnjak I, Flitney DE. 2004. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage. 23:S208–S219. [DOI] [PubMed] [Google Scholar]
- Strait CE, Sleezer BJ, Hayden BY. 2015. Signatures of value comparison in ventral striatum neurons. PLoS Biol. 13:e1002173. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sunstein CR. 2017. Default rules are better than active choosing (Often). Trends Cogn Sci. 21:600–606. [DOI] [PubMed] [Google Scholar]
- Suzuki S. 1997. Effects of number of alternatives on choice in humans. Behav Processes. 39:205–214. [DOI] [PubMed] [Google Scholar]
- Suzuki S. 1999. Selection of forced-and free-choice by monkeys (Macaca fascicularis). Percept Mot Skills. 88:242–250. [Google Scholar]
- Thornton JW, Jacobs PD. 1971. Learned helplessness in human subjects. J Exp Psychol. 87:367. [Google Scholar]
- Tricomi E, Fiez JA. 2012. Information content and reward processing in the human striatum during performance of a declarative memory task. Cogn Affect Behav Neurosci. 12:361–372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang KS, Smith DV, Delgado MR. 2016. Using fMRI to study reward processing in humans: past, present, and future. J Neurophysiol. 115:1664–1678. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weissenbacher A, Kasess C, Gerstl F, Lanzenberger R, Moser E, Windischberger C. 2009. Correlations and anticorrelations in resting-state functional connectivity MRI: a quantitative comparison of preprocessing strategies. Neuroimage. 47:1408–1416. [DOI] [PubMed] [Google Scholar]
- White RW. 1959. Motivation reconsidered: the concept of competence. Psychol Rev. 66:297. [DOI] [PubMed] [Google Scholar]
- Woolrich MW, Behrens TE, Beckmann CF, Jenkinson M, Smith SMJN. 2004. Multilevel linear modelling for FMRI group analysis using Bayesian inference. Neuroimage. 21:1732–1747. [DOI] [PubMed] [Google Scholar]
- Worsley K. 2001. Statistical analysis of activation images. Funct MRI. 14:251–270. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.