Reward salience and risk aversion underlie differential ACC activity in substance dependence

Abstract

The medial prefrontal cortex, especially the dorsal anterior cingulate cortex (ACC), has long been implicated in cognitive control and error processing. Although the association between ACC and behavior has been established, it is less clear how ACC contributes to dysfunctional behavior such as substance dependence. Evidence from neuroimaging studies investigating ACC function in substance users is mixed, with some studies showing disengagement of ACC in substance dependent individuals (SDs), while others show increased ACC activity related to substance use. In this study, we investigate ACC function in SDs and healthy individuals performing a change signal task for monetary rewards. Using a priori predictions derived from a recent computational model of ACC, we find that ACC activity differs between SDs and controls in factors related to reward salience and risk aversion between SDs and healthy individuals. Quantitative fits of a computational model to fMRI data reveal significant differences in best fit parameters for reward salience and risk preferences. Specifically, the ACC in SDs shows greater risk aversion, defined as concavity in the utility function, and greater attention to rewards relative to reward omission. Furthermore, across participants risk aversion and reward salience are positively correlated. The results clarify the role that ACC plays in both the reduced sensitivity to omitted rewards and greater reward valuation in SDs. Clinical implications of applying computational modeling in psychiatry are also discussed.

Highlights

• •We test model-based predictions regarding ACC function in substance dependence.
• •Increased risk aversion contributes to altered ACC function in substance dependence.
• •Contributes to the nascent field of computational psychiatry

1. Introduction

The medial prefrontal cortex (mPFC), especially the dorsal anterior cingulate cortex (ACC), is recognized as a key brain region involved in cognitive control and decision making. The region focused on here includes the rostral cingulate zone of Picard & Strick (2001) and extends dorsally and caudally into the pre-SMA (Nee et al., 2011). Neuroimaging studies have explored multiple hypotheses to explain the involvement of ACC in monitoring goal-directed behaviors, including conflict monitoring (Botvinick et al., 1999), error detection (Brown and Braver, 2005), allocation of attention resources (Carter et al., 1998; Whalen et al., 1998), action–outcome evaluation (Alexander and Brown, 2011; Gehring and Willoughby, 2002; Hyman et al., 2013; Kennerley et al., 2006), predicting task difficulty (Brown and Braver, 2005), and updating predictions of expected cognitive demands (Sheth et al., 2012). Specifically, the ACC is known to play a key role in decision making (Botvinick et al., 1999; Carter et al., 2000; Gehring and Knight, 2000; Li et al., 2010; Paulus and Frank, 2006). Depending on the decision making paradigms, the ACC has shown to be a critical signal in evaluating the perceptions of risk and predicted reward (Alexander and Brown, 2010), the anticipation of risk (Fukui et al., 2005; Krawitz et al., 2010), driving loss avoidance (Fukunaga et al., 2012; Magno et al., 2006), as well as learning the likely consequences of risky behavior (Brown and Braver, 2005; Brown and Braver, 2007).

1.1. ACC deficits and risk-taking behaviors: competing hypotheses

Disturbances in mPFC/ACC functioning are widely believed to contribute to deficits in top-down cognitive control processes across a multitude of psychiatric disorders (Carter et al., 1997; Drevets et al., 1997; Gehring et al., 2000; Schmidtke et al., 1998; Shin et al., 2001; Yucel et al., 2003). Addictions research, in particular, has widely reported deficits in cognitive control to be associated with negative outcomes for substance dependent individuals (SDs), including decreased performance on tasks thought to involve the prefrontal cortex (Bolla et al., 2001; Fishbein et al., 2005; Gowin et al., 2014; Hester et al., 2009; Hester and Garavan, 2004; Kaufman et al., 2003), poor treatment outcomes (Charlet et al., 2014; Steele et al., 2014), treatment compliance (Streeter et al., 2008), as well as an increased likelihood of relapse (Clark et al., 2014; Marhe et al., 2013; Paulus et al., 2005; Stewart et al., 2014). Recent work also has shown ACC dysregulation during reward-seeking behavior to be associated with excessive alcohol consumption, as seen in the Balloon Analog Risk Task (Bogg et al., 2012). These findings are consistent with neuroimaging and modeling work showing ACC as learning the likelihood of an error in a change signal task (Brown and Braver, 2005) as well as the potential consequences of risky behavior (Brown and Braver, 2007; Brown and Braver, 2008; Jahn et al., 2011). Thus, disengagement of the ACC may contribute to risky behaviors that may include substance use (Fukunaga et al., 2013).

Conversely, increased ACC activity has also been associated with substance dependence in some circumstances. A wealth of evidence indicates that substance-related cues reliably engage ACC for SDs during cue-induced craving (Childress et al., 1999; Garavan et al., 2000; Goldstein et al., 2007b; Heidbreder, 2011; Maas et al., 1998), even after periods of non-substance use (Ciccocioppo et al., 2001). In some cases, increased ACC activity related to cognitive control was observed for SDs, in tasks with drug-cue distractors (Luijten et al., 2011; O'Leary et al., 2000), which can be interpreted as the need for increased control in order to perform a behavioral task. Greater control may be needed, for example, to overcome salient distractors (Botvinick et al., 2001), or to compensate for decreased efficiency of processing (Poldrack, 2015). Drugs with a high potential for dependence tend to engage the dopaminergic system, and result in strong activation of ACC (Breiter et al., 1997; O'Leary et al., 2000), which is densely innervated by midbrain dopamine neurons (Oades and Halliday, 1987). Thus, greater ACC activity may also result from abnormally high dopaminergic reinforcement signals (Redish, 2004).

Together, these competing findings put forth two potential and possibly related factors underlying ACC involvement in substance dependence. First, the risk-seeking hypothesis suggests that substance-dependence related disengagement of ACC in cognitive control tasks may contribute to a lack of sufficient inhibitory control needed to prevent automatic or habitual behaviors related to substance use, resulting in responses that increase risk. Second, the reward salience hypothesis, suggests that increased reward salience of substance use and substance-related cues may co-opt control processes such that behaviors related to substance dependence become subjectively more valuable. This is a critical distinction from risky behavior in that, rather than implying a lack of control, it implies over-engagement of control processes related to acquiring and using a substance. As a whole, the above literature also suggests an alteration of cognitive control processes in SDs in which attentional bias effects (Field and Cox, 2008; Hester et al., 2006; Hester and Garavan, 2009) lead to ACC over-activation when SDs process drug-related tasks, but at the cost of impaired processing of non-drug related tasks.

1.2. Applying computational models of ACC in substance dependence

Despite the acknowledged link between ACC function and behavior, neuroscientific research has thus far failed to converge on the exact role of ACC function in supporting behavior related to substance dependence. This lack of clarity regarding ACC involvement in substance dependence may partially be a result of uncertainty surrounding ACC function itself. As noted above, ACC has variously been implicated in a number of different functions, including monitoring and processing behavioral error (Gehring et al., 1990), detecting behavioral conflict (Botvinick et al., 2001), tracking environmental volatility (Behrens et al., 2007), predicting the likelihood of error (Brown and Braver, 2005), learning the value of actions (Rudebeck et al., 2008; Walton et al., 2004), and many more. A recent computational model of ACC (Alexander and Brown, 2011) recasts the role of the region as learning to predict the likely consequences of an action, regardless of affective valence, and signaling when an expected outcome fails to occur. The predicted response–outcome (PRO) model comprehensively accounts for a wide range of data from fMRI, EEG, and single-unit neurophysiology studies involving ACC under a single unifying framework (Alexander and Brown, 2011).

Using the PRO model, we attempt to distinguish between the two general hypotheses described above regarding the influence of substance dependence on ACC activity in a cognitive control task. First, we formalize the risk-seeking and reward salience hypotheses using concepts from the judgment and decision-making literature. Specifically, we adopt the notion of risk frequently deployed in the context of decision-making under uncertainty in which risk-aversion and risk-seeking reflect the degree of concavity or convexity, respectively, of a utility function. This definition of risk-aversion is distinct from other possible meanings of risk-aversion, under which risk may be defined as the probability of loss, the uncertainty or variance surrounding a prospective outcome, or the tendency to engage in behaviors that may result in harmful outcomes (Krawitz et al., 2010). We address these various alternate definitions of risk in greater detail in the discussion.

Next, using simulations of the PRO model performing a cognitive control task, we derive a priori predictions regarding the activity of ACC while manipulating model parameters associated with reward salience and risk-seeking. Based on these simulations, detailed below, we expect that under the risk-seeking hypothesis, ACC activity related to reward magnitude and error likelihood will be higher in substance dependent individuals (SDs) relative to control participants, while under the reward salience hypothesis, ACC activity for SDs will be greater for increased reward magnitude and lower for increased error likelihood. We then test these predictions using fMRI to record brain activity in SDs and non-substance dependent individuals (non-SDs) while performing an Incentive Change Signal Task (ICST). Lastly, in a second set of model-based analyses, we fit the PRO model to observed fMRI data by estimating the best-fit reward salience and risk parameters on a per-subject basis in order to identify differences between SDs and non-SDs in attention to reward or risk attitudes.

2. Methods

Recruiting and experimental procedures were approved by the Indiana University Bloomington Institutional Review Board. Several components of the methods have been reported in our previous study (Alexander and Brown, 2010).

2.1. Participants

A total of 49 subjects took part in the present study and provided written informed consent. All subjects were required to be at least 18 years of age, right-handed, and to meet standard health and safety requirements, including no history of neurological problems or claustrophobia, weigh less than 440 lbs, and have no metallic implants, for entry into the magnetic resonance imaging scanner. They were paid $25/h for participation, plus performance bonuses (see below) averaging approximately $6.70.

2.1.1. Non-substance-dependent (non-SD) control group

Non-SD subjects (n = 24) were initially recruited for an earlier version of this study, and the results have been published previously (Alexander and Brown, 2010). Here we use the same data to compare against the SD participants. The non-SD participants are therefore representative of the general population with a relatively low rate of substance dependence; however, the original study did not explicitly exclude for current or past alcohol abuse or other substance abuse or dependence. By contrast, all subjects in the SD group were subject to a separate exclusion criteria to ensure that they met the criteria for substance dependence. We further address the potential limitation of the differing screening procedures in the Discussion section. The control subjects were run between late July and late October of 2007.

2.1.2. Substance-dependent (SD) group

SD subjects (n = 25) were recruited using advertisements placed around campus and the Bloomington community (see Finn et al. (2009) for specifics about the recruitment strategy). The initial inclusion criteria for the SD group required participants to meet additional eligibility requirements: (a) be between the ages of 18–30 years, (b) be able to read and speak English (whether as a native or second language), (c) have at least a sixth grade level of education, (d) have consumed alcohol, (e) have no reports of suffering from any serious head injuries, (f) have no major cognitive impairments and; (g) have no history of psychotic symptoms.

Individuals who met this preliminary criteria were administered a diagnostic interview, using the Semi-Structural Assessment for the Genetics of Alcoholism (SSAGA) (Bucholz et al., 1994) to ascertain diagnoses for substance use disorders using DSM-IV diagnostic criteria (American Psychiatric Association, 2000). The SSAGA is an extensively used diagnostic interview designed to ascertain diagnoses within the DSM system (DSM-III and DSM-IV), the ICD-10 system, as well as the RDC system. The SSAGA was administered by senior doctoral students in Clinical Psychology and supervised by a licensed clinical psychologist.

All SD subjects met DSM-IV criteria for either: (a) alcohol dependence and no drug abuse (n = 3); (b) marijuana dependence and no polydrug abuse (n = 7); (c) marijuana or other drug dependence (apart from alcohol) and polydrug abuse (n = 15). In this mixed group, participants met for separate diagnoses for both marijuana dependence, as well as dependence for a separate drug type. Some of the subjects in (b) and (c) also met DSM-IV criteria for alcohol dependence. Due to the relatively low numbers of individuals reported for each subgroup, subjects were combined during the analysis, enabling the maximization of statistical power. All SD participants were asked to refrain from using either alcohol or drugs for at least 12 h prior to their scheduled fMRI session. The SD subjects were run over a 10-month period between mid-April 2008 and mid-February 2009. Both control and substance dependent subjects were run in the same facility and at similar times of day, and during overlapping months of the year.

It is important to note that the SD subjects were recruited from the general population and were not currently in treatment for substance dependence. Apart from current marijuana, alcohol, or tobacco dependence, all substance dependence is past rather than current. Subjects were allowed to smoke prior to the session if they desired. We did not breathalyze or urine screen the subjects, but anyone who showed signs of current intoxication was not scanned. None of our alcohol dependent subjects were experiencing classic withdrawal symptoms, and our assessments revealed furthermore that the alcohol dependent subjects had never experienced classic withdrawal symptoms. The reason is that our subjects were relatively young and were therefore early in their careers in alcohol dependence, where symptoms and drinking levels wax and wane. None of the subjects drank to intoxication every day, and many did not drink at all some days. Thus, alcohol-dependent subjects were generally not experiencing withdrawal symptoms during the scan session.

With regard to the polydrug subgroup, each participant met the criteria for a varying number of drug types and severity (abuse versus dependence) unless otherwise specified, the drug type indicated is assumed to have been met for dependence: Participant 1 (Alcohol, Marijuana, Stimulant, Nicotine, Hallucinogen (abuse)); Participant 2 (Marijuana, Stimulant, Nicotine, Opioid); Participant 3 (Alcohol, Marijuana, Stimulant, Sedative (abuse)); Participant 4 (Alcohol (abuse), Stimulant, Opioid); Participant 5 (Alcohol (abuse), Marijuana, Nicotine, Hallucinogen (abuse)); Participant 6 (Marijuana, Stimulant, Nicotine, Hallucinogen, Opioid (abuse), Sedative (abuse)); Participant 7 (Alcohol; Marijuana, Opioid (abuse), Sedative); Participant 8 (Alcohol, Marijuana, Stimulant (abuse)); Participant 9 (Alcohol, Marijuana, Nicotine, Opioid); Participant 10 (Alcohol, Marijuana, Stimulant, Opioid, Hallucinogen (all abuse)); Participant 11 (Alcohol (abuse), Marijuana, Stimulant, Opioid, Nicotine); Participant 12 (Alcohol, Marijuana, Stimulant); Participant 13 (Alcohol (abuse), Marijuana, Stimulant (abuse), Nicotine); Participant 14 (Alcohol (abuse), Marijuana, Stimulant) and Participant 15 (Alcohol, Marijuana, Nicotine).

While the control group was not explicitly screened to exclude drug dependent individuals, we assessed self-reports of smoking (smoker status and cigarettes per day) and binge drinking in all subjects. Alcohol consumption self-reports were collected in the course of administering the domain specific inventory of risk-taking (DOSPERT) (Weber et al., 2002), with the question of how likely the subject is to drink heavily at a social function. Responses were on a Likert scale of 1–5, with 5 meaning very likely. For smoking, controls were significantly less likely to smoke (4 controls, 16 SD, p < 0.02, Fisher exact test). For drinking, controls were significantly less likely to drink than SD subjects (controls mean = 2.64; SD mean = 4.18; t(45) = 4.40, p < 0.00007, two tail). Across the general subject population, alcohol use disorders occur in 15–22% of subjects (Blanco et al., 2008; Slutske, 2005; Wu et al., 2007), while drug use disorders occur in 5–7% of subjects (Blanco et al., 2008). Overall, while the rates of alcohol and drug use disorders are likely not zero in the control group, they are significantly lower than in the SD group.

2.2. Design and procedure

2.2.1. Behavioral task

The Incentive Change Signal Task (ICST) (Brown and Braver, 2007) is a modified version of the change signal task (Brown and Braver, 2005) and was implemented in E-Prime (Psychology Software Tools, Pittsburgh, PA). The ICST consisted of four phases: color cue, target, response, and feedback (see Fig. 1). At the beginning of each trial two horizontal dashes were displayed in the center of the screen. Dashes were one of four colors: white, brown, yellow, or light blue. Each color was paired with one of the four possible combinations of error likelihood (high and low) and average reward magnitude (high and low). These pairings were counterbalanced across all participants, and the pairings were constant across all trials for an individual participant. Trials were presented pseudo-randomly. After the dashes were displayed for 1000 ms, an angle brace appeared to the right or left of the dashes, forming an arrow pointing either left or right (48 pt font). The direction of the arrow indicated which response the participant was to make, either with the left or right index finger. On change signal trials (1/3 of all trials), an additional arrow (96 pt font) appeared above the first arrow and pointing in the opposite direction, indicating that the participant was to cancel the initial response and make a response according to the second arrow. The stimuli remained visible for 1000 ms after the appearance of the first arrow. The change signal delay (CSD) between the onset of the initial arrow and the second arrow was adjusted by an asymmetric stairstep algorithm to maintain target error rates, and the CSD was adjusted independently for each of the four colors. For the low error likelihood (EL) conditions, the CSD was adjusted to achieve an error rate of 5% on change signal trials, while an error rate of 50% was maintained for the high error likelihood conditions. Subjects were not informed beforehand that the CSD would be adjusted in this way. On each change trial, the CSD was increased for a correct trial, while incorrect trials decreased the CSD. After presentation of the stimuli and expiration of the response deadline, the screen was blank for 500 ms, after which visual feedback was provided to the participants for 1000 ms. For correct trials, feedback consisted of the word ‘Correct’ and 4 digits indicating how many points the participant earned for the trial. For incorrect trials, participants saw the word ‘Incorrect’ in addition to the number of points earned. The number of points earned on each trial depended both on the outcome (correct or incorrect) of the trial as well as the average reward magnitude (RM) condition. For the high RM condition, subjects earned 2000 pt for a correct trial and 1000 pt for an incorrect trial, while in the low RM condition subjects earned 1000 pt for a correct trial and 0 pt for an incorrect trial. Participants were informed that their points were to be converted directly to a cash payment at the end of the session. Points were converted at the rate of 1000 pt for each US $0.01. Participants were not informed of the conversion rate of points to dollars prior to participation, nor were they given direct information regarding their accumulated point total. We found in pilot studies that subjects performed with greater motivation for large amounts of points, with conversion factors revealed after the session, than for the equivalent relatively small monetary payment. After feedback, the screen remained black for a minimum of 1500 ms until the start of the next trial. Intertrial intervals (ITIs) were jittered by adding 0, 2000, 4000, or 6000 ms (3 TRs) to the ITI. Jitter delays were chosen by a weighted random selection of each of the possible durations; the weights for each of the jitter durations were 30, 12, 5 and 2, respectively, allowing for efficient estimation of the hemodynamic response function (HRF) (Burock et al., 1998).

Fig. 1.

Change signal task and model predictions. In the change signal task (A), participants are presented with an initial cue indicating a response to be made. On a subset of trials, a second cue is presented after a variable delay, indicating that the subject should cancel the initial response and instead make the alternate response. The color of the cue implicitly indicates the likelihood of error (2 levels) and reward magnitude (2 levels). Simulations of the PRO model B were used to generate a priori predictions of individual differences in ACC activity related to reward salience and aversion to risk. As sensitivity to reward (indicated by the parameter λ, cf. Fig. 2) increases, error likelihood effects attenuate while reward magnitude effects increase. As risk-aversion (γ, cf. Fig. 2) increases, both error likelihood effects and effects of reward magnitude increase.

Participants performed 6 blocks of 82 trials per block in the scanner. Participants were trained on the task prior to scanning in order to familiarize them with the task instructions, but not the specific reward magnitude and error likelihood conditions. Training typically consisted of fewer than the 82 trials comprising a single block. Subjects learned the payoff amounts and probabilities associated with each color cue condition solely by experience while performing the task in the scanner, as in previous studies (Brown and Braver, 2005; Brown and Braver, 2007). Differences in BOLD signals due to effects of reward magnitude and error likelihood are therefore the result of experience with the task during scanning, and not previous training.

In the task design, reward magnitude was manipulated by adding 1000 pt to the outcomes such that a correct response in the high reward magnitude condition was worth 1000 pt more than a correct response in the low reward magnitude condition, and, similarly, an error was worth 1000 pt more in the high reward magnitude condition than in the low reward magnitude condition. Average reward is commonly calculated as the sum of the probability of each potential outcome multiplied by the value of that outcome. In the current task, manipulation of error likelihood necessarily affects the actual expected value of each condition. In high error likelihood conditions, a participant is more likely to commit an error, leading to a lower average reward than in the low error likelihood condition. Critically, however, changes in average reward are the same across conditions: the difference between the average reward (high RM and low RM) in the low error likelihood conditions is the same as the difference in the high error likelihood condition (Fig. 1). Thus, we can independently manipulate error likelihood and reward magnitude.

We used the domain specific inventory of risk-taking (DOSPERT (Weber et al., 2002)), to assess the likelihood of an individual engaging in risky behaviors in 5 domains: financial decisions (investing and gambling were measured separately), ethical choices, health/safety, social interaction, and recreation. Most pertinent to our analysis were two specific domains, financial decisions and health/safety risk taking, and thus the remaining domains were not considered in this current analysis. We used the 40-item DOSPERT questionnaire, which consisted of questions, for example, inquiring about the likelihood of an individual “drink[ing] heavily at a social function,” “engaging in unprotected sex,” “betting a day's income at high-stake poker game,” and “gambling a week's income at a casino” (Weber et al., 2002).

2.3. fMRI analysis

2.3.1. Imaging acquisition and preprocessing

Functional images were collected on a Siemens Magnetom Trio at 3.0 Tesla MRI scanner at the Imaging Research Facility of the Indiana University Bloomington Campus. Functional image slices were tilted 30° toward the coronal plane from the AC–PC line for whole-brain coverage (EPI, 33 slices, 3 mm slice thickness, TR = 2000 ms, TE = 25, flip angle = 70, FOV = 220 × 220 mm, 64 × 64 voxel in-plane resolution, voxel size = 3.4375 mm by 3.4375 mm by 3 mm = 35.4492 mm³). T1-weighted structural images for each participant also were acquired at the end of each session using three-dimensional MP-RAGE imaging (160 sagittal slices, 1 mm slice thickness, TR = 2300 ms, TE = 3.93, flip angle = 12, pixel width inplane = 0.5 mm).

Preprocessing was done using SPM5 (Wellcome Trust Centre for Neuroimaging, 2005) except where otherwise specified. Functional data were spike-corrected on a voxel-by-voxel basis to reduce the impact of artifacts using AFNI's 3dDespike. The structural scan was skull-stripped using FSL's BET2 with default parameters (Péchaud et al., 2006). The functional images were slice-timing corrected using sinc-interpolation (Oppenheim et al., 1999), motion corrected by means of a least-squares 6-parameter rigid-body transformation, and coregistered with the structural scan. Once the structural scan was normalized to the SPM MNI template, the normalized images were smoothed with an 8 mm³ FWHM isotropic Gaussian kernel.

2.3.2. Intrasubject analysis

Event-related responses were estimated using a general linear model approach and analyses conducted using SPM5 and the Marsbar (Brett et al., 2002) toolkit for ROI analyses. A general linear model (GLM) was estimated for each subject using a total of 17 regressors: a constant term, 6 regressors for movement, and 10 regressors for experimental conditions. Eight regressors were used to model correct trials for all combinations of levels of high vs. low reward magnitude, high vs. low error likelihood, and change vs. go trials (e.g., trials in which a change signal was either presented or not presented). Events were time-locked to the onset of each trial (appearance of angle brace indicating which response the subject should make) and were modeled as having duration of 0 s (as is standard in SPM). Error trials were modeled by two regressors, one for errors made for change trials, and another for errors committed when no change signal was presented or when no response was made. Beta values for model regressors were estimated using the SPM canonical HRF.

2.3.3. Group analysis

Analyses for main effects, interactions, and pairwise comparisons were done at the 2nd-level (random effects), and performed only for correct go trials at the whole-brain level. Planned analyses included tests for differences in error likelihood effects (correct/go/high EL − correct/go/low EL) and differences in effects of reward magnitude (correct/go/high RM − correct/go/low RM) between groups (Controls–SD). These contrasts provide for direct tests of our a priori predictions, as derived below from the computational modeling. The threshold for voxels to be included in cluster-level significance testing was set to p < 0.01. This threshold is more liberal than the threshold of p < 0.001 recommended by Woo et al. (2014). However, we note that this threshold was determined using whole brain analyses in which large clusters of voxels crossing functional or anatomical boundaries may be formed by chance. Given that our analyses are informed by a priori predictions specifically regarding ACC, and that analyses pertaining to these predictions are restricted to ACC, the possibility that clusters identified in these analyses as being significant due to trans-region cluster formation is reduced, and thus a more liberal threshold is warranted. In order to rule out the possibility that the results reported below were the result of using a predefined anatomical ROI in conjunction with a more liberal cluster threshold, we also conducted whole brain analyses using the more typical cluster threshold of 0.001. Except where noted, regions of interest for additional analyses were selected by the peak area of activation for clusters of activation that passed familywise error (FWE) cluster correction for our planned analyses.

2.4. Computational methods

2.4.1. Generating model predictions

In the Introduction section, we outlined two possible accounts for behavior related to substance dependence, namely that SDs may have increased attention to rewarding outcomes versus aversive outcomes, or that SDs may have reduced risk aversion relative to non-SDs. Here we derive predictions regarding ACC activity for the ICST using the PRO model of ACC (Alexander and Brown, 2011). The PRO model learns predictions of the likely outcomes of actions. In the ICST, there are four action/outcome conjunctions: Go/Correct, Go/Error, Change/Correct, & Change/Error. In previous simulations, it was assumed that the occurrence of an outcome was a binary event — either the outcome occurred or it did not. In order to simulate effects of reward magnitude, potentially modulated by risk attitudes or reward sensitivity, we model outcomes as continuous numbers on the interval [0,1] using an exponential utility function (Fig. 2 (Pratt, 1964)) of the following form for correct trials:

$$U\left(\mathit{correct}\mathit{trials}\right)=\lambda \times \left\{\begin{array}{c}\hfill 1-e^{-\gamma x}\mathrm{if}\gamma >0\hfill \\ \hfill \frac{x}{2000}\mathrm{if}\gamma =0\hfill \\ \hfill e^{\gamma x}\mathrm{if}\gamma <0\text{.}\hfill \end{array}\right)$$

Fig. 2.

Example of utility functions as related to risk preferences.

Here x is the total points received by a subject for a correct trial (2000 in High RM trials, 1000 in Low RM trials), and γ is a utility parameter reflecting risk-attitude. For γ > 0, the utility function is concave, reflecting risk-aversion, while for γ < 0 the utility function is convex, reflecting risk-seeking. In this latter case, U was rescaled such that it fell between 0 and 1 by dividing U by max(U). Finally, λ represents the balance between sensitivity to reward and reward omission. For λ = 1 the utility function attends only to gains while reward omission (described below) is ignored. Conversely, for λ = 0, rewards are ignored and only omission of reward influences learning in the model. Error trials were modeled as the difference in utility of the total number of points possible on a trial and the total number of points actually received (1000 in High RM trials, 0 in Low RM trials):

$$U\left(\mathit{error}\mathit{trials}\right)=\left(1-\lambda \right)\times \left\{\begin{array}{c}\hfill \left(1-e^{-\gamma x\left(\mathit{win}\right)}\right)-\left(1-e^{-\gamma x\left(\mathit{actual}\right)}\right)\mathrm{if}\gamma >0\hfill \\ \hfill \frac{x\left(\mathit{win}\right)-x\left(\mathit{actual}\right)}{2000}\mathrm{if}\gamma =0\hfill \\ \hfill e^{\gamma x\left(\mathit{win}\right)}-e^{\gamma x\left(\mathit{actual}\right)}\mathrm{if}\gamma <0\text{.}\hfill \end{array}\right)$$

We simulated the PRO model on the ICST task while parametrically varying λ (keeping γ constant at 0, indicating indifference to risk) and γ (keeping λ constant at 0.5, indicating equal sensitivity to reward and reward omission) in order to derive predictions regarding ACC activity (Fig. 1B). All other parameters were identical to those previously reported (Alexander and Brown, 2011).

A total of 4 response–outcome (RO) conjunctions are simulated, representing the possible actions available to the model (Go and Change) and the possible feedback as a result of selecting an action (Correct and Incorrect), as in previous model simulations (Alexander and Brown, 2011). Single units in the model represent each of the possible RO conjunctions and constitute a feedback signal allowing the model to learn to predict likely outcomes. Whereas outcomes were modeled as binary events in the original simulations of the PRO model, here outcomes were modeled as the numerical value for the utility function parameterized by λ and γ. Thus, when the model responded correctly to a change signal, the level of activity for the unit in the model signaling a Change/Correct conjunction was set to the value computed from the above utility functions. Only 1 feedback unit was active for each trial.

Model-derived predictions for between group differences for RM effects (High RM − Low RM) and EL effects (High EL − Low EL) are summarized in the right panels of Fig. 1B. Our two principal hypotheses, that SDs may be either more risk-seeking or more sensitive to reward relative to our control group can be differentiated by EL effects: under the risk-seeking hypothesis, SDs should show greater EL effects than controls, while under the reward salience hypothesis, SDs should show weaker EL effects than controls. For completeness, we also considered possible scenarios in which SDs were more risk averse than controls, as well as more sensitive to reward omission.

In order to derive model predictions, it was assumed in our simulations that non-substance dependent controls would be equally sensitive to reward and reward omission (λ = 0.5), and neutral in their risk attitudes (γ = 0). These assumptions are likely invalid, as even in healthy populations, individuals tend to be risk averse and more sensitive to losses than gains (Kahneman and Tversky, 1979). We examine the validity of these assumptions by fitting the model to individual activation patterns (described below).

2.4.2. Model fitting

In a second line of analyses, we fit the activity of the PRO model to fMRI data from individual subjects, as well as group data, in order to estimate parameters λ and γ of the utility functions described above. Quantitative model fits of fMRI data present a number of methodological challenges (Ashby and Waldschmidt, 2008). A common approach to model-based fMRI analysis is to train a computational model on the sequence of events observed by a subject and to regress the resulting model activity against fMRI data (Jahn et al., 2014; O'Doherty et al., 2003) in order to identify voxels that correlate well with the predictions of the model.

Our approach is similar in that our goal is to correlate model activity with observed fMRI activity. However, rather than trying to identify voxels that correlate with model activity for a given model parameterization (often derived from fits to behavioral data), our aim is to find the model parameterization that reproduces the observed pattern of activity within a selected set of voxels.

Accordingly, we identify a cluster in dACC showing significantly greater error likelihood effects in controls relative to SD, namely for the contrast ((Control/HEL − Control/LEL) − (SD/HEL − SD/LEL)). Note that by selecting this region on the basis of the between-groups comparison of EL effects, we introduce a potential bias into the model fit. Specifically, of the hypothesized influences on ACC activity outlined in Fig. 1B, two of them, increased risk aversion and increased reward salience in SDs, are compatible with the between-groups EL effects we observe in our data, and therefore the model fit is biased to recover parameter values consistent with either increased reward salience or increased risk aversion. However, we note that the direction of between-groups RM effects is different for these two hypotheses: increased reward salience results in larger RM effects, while increased risk aversion results in decreased RM effects (Fig. 1B). It is therefore not specified by fitting the model to fMRI data from the ROI chosen based on between-groups EL effects which parameter estimate will differ between groups. That is, even though the model fit will be biased to find either that the risk-aversion parameter or the reward salience parameter is greater for SDs relative to controls, the fit is agnostic with regard to which of those parameters underlie the observed between-groups EL effects. It is possible that reward salience alone is responsible for group differences, in which case estimates for the risk-aversion parameter may not differ between SDs and controls. Alternatively, risk-aversion alone may underlie group differences, while estimates for rewards salience may not differ between groups. By fitting the model to regions showing between group differences in EL effects, our goal is twofold: first, to elaborate on the relationship between reward salience and risk preferences within ACC, and second, to investigate our assumptions of risk-neutrality and reward omission indifference in our control group.

Using MarsBar (Brett et al., 2002), we calculated the percent signal change for each subject in this region for all RM and EL conditions for only Go/Correct trials. For each subject, we fit the PRO model to the difference in percent signal change for all pairwise comparisons as well as the main effects of RM and EL. The model was trained on the sequence of trials and outcomes observed by that subject, and model activity was calculated as the difference between prediction and outcome (i.e. negative surprise (Alexander and Brown, 2011)) for the first 0.5 s of the trial, averaged over all trials for each condition. Free parameters in the model were the degree of risk-aversion/risk-seeking (λ), reward/omission sensitivity (γ), and a scaling factor. This last parameter had no influence on the pattern of activity predicted by the model, and was included in order to give fMRI and model activity the same (arbitrary) units.

As noted in the section describing recruitment and screening of participants, members of the control group were not explicitly screened for substance dependence, leading to the possibility that our control group is contaminated by individuals with undiagnosed substance dependence. Generally, the use of a contaminated control group will lead to an increase in type II errors due to the dependent variable for contaminated individuals in the control group being more similar to the population of interest than the legitimate controls. Nevertheless, we cannot conclusively state, based on the values of the dependent variables (in this case, estimated model parameters), which of our control group participants may have qualified for a diagnosis of substance dependence and, consequently, should have been excluded from the study; it is possible that undiagnosed SD individuals in the control group are contributing to any observed differences between estimated model parameters for each group. In order to address this weakness of the study, we conducted a permutation test on the estimated model parameters for each group using the rate of dependence on alcohol and other substance in the population to inform our estimate of the likely number of participants in the control group with substance dependence.

The permutation test was conducted in two steps. First, we calculated the probability of a single person being dependent based on the frequency observed in the populations of 15–22% and 5–7% for alcohol and other substances, respectively. This probability was calculated using the maximum of the range reported for each group as follows:

$$\mathrm{P}\left(\mathrm{S}\mathrm{D}\right)=\mathrm{P}\left(\mathrm{AD}\right)+\mathrm{P}\left(\mathrm{O}\mathrm{D}\right)‐\mathrm{P}\left(\mathrm{AD}\right)\mathrm{P}\left(\mathrm{O}\mathrm{D}\right)$$

Here AD indicates alcohol dependence and OD indicates dependence on another substance. Using the maximum frequencies, this gives a probability of a random individual being substance dependent of 0.2746. Second, probability that our control group of 24 subjects contains n substance-dependent individuals is given by the binomial distribution:

$$P\left(X=n\right)=\left(\begin{array}{c}\hfill 24\hfill \\ \hfill n\hfill \end{array}\right)p^n{\left(1-p\right)}^{24-n\text{.}}$$

In the second step, for each possible value of n, we conducted t-tests between estimated model parameters for the SD group and a control group in which n participants were excluded. The identity of the excluded participants was permuted for each value of n such that all possible combinations were tested. For example, in the case of n = 1 (indicating a single contaminated control participant), there are 24 possible permutations, while for the case of n = 12, there are 2,704,156 possible permutations. For each value of n, the probability of rejecting the null hypothesis was calculated as the number of times the null hypothesis was rejected over the total number of permutations for that value of n. Finally, we calculated the probability of rejecting the null hypothesis overall as the sum over n of the probability of rejecting the null hypothesis for a given value of n times the probability of n having that value:

$$P\left(H_0\ne H_1\right)={\displaystyle \sum _{n}P\left(H_0\ne H_1\left|n\right)\right)P\left(X=n\right)}\text{.}$$

3. Results

3.1. Group demographics

Demographic characteristics of the two groups are given in Table 1. Pearson's chi-square tests showed no significant differences between the non-SD and SD groups in terms of age, gender, socioeconomic status (SES; based on father's level of education), ethnicity, and race.

Table 1.

Data are represented as means plus/minus standard errors.

		Non-SD group	SD group	Analysis
		(N = 24)	(N = 25)	χ2	p
%		48.9	51.0
Age		22.54 ± 0.80	21.76 ± 0.44
Gender	Female	13	12	0.19	0.67
	Male	11	13
Education (level)	Graduate school	7	6	1.48	0.69
	Standard college	7	5
	High school graduate	7	13
	Other	3	1
Substance use history
	Alcohol dependence only	a	3
	Marijuana dependence only	a	7
	Drug dependence/polydrug abuse	a	15
DOSPERT likelihood scoresb
	Gambling	6.00	8.00		<0.04
	Health & safety risk-taking	20.44	24.58		<0.005
	Drinking heavily	2.64	4.18		<0.00007

^aDetailed substance use histories not obtained for the non-SD group (see the Limitations and future questions section for more details).

^bSelf-report questionnaires were not available for one subject in the SD group.

3.1.1. Behavioral results

A 4-way ANOVA (RM × EL × GROUP × TIME) indicated no main effect of either RM condition (F(3,777) = 0.25, p = 0.621) nor EL condition (F(3,777) = 1.6, p = 0.206) on RTs in Go/Correct trials. To investigate potential learning effects, trials were separated into 4 bins based on trial number (i.e. trials during each quarter of the experiment were included in the same bin); however, no main effect of trial number on RT was observed (F(3,777) = 0.17, p = 0.92). However, RTs for controls (mean = 729.88 ms, standard deviation = 55.81 ms) and RTs for SDs (mean = 750.86 ms, standard deviation = 64.48 ms) were significantly different (F(3,777) = 22.67, p < 0.001), ACC activity has previously been observed to correlate with reaction time (Carp et al., 2010; Grinband et al., 2011), suggesting that differences in mean ACC activity between groups across all Go/Correct trials may be related to differences in RT. In the present study, however, our planned fMRI analyses examine between group differences in the RM and EL effects. Any differences in the BOLD signal observed for these analyses would not be expected to correlate with between-group differences in RT, but might be related to within-group RT differences. A 2-way ANOVA was conducted to test for a main effect of RM or EL on RT within each group. We found no significant effect of RM (F(1,93) = 0.23, p = 0.63) nor EL (F(1,93) = 0.2, p = 0.66) on RT for controls. Similarly, no significant effect of RM (F(1,97) = 0.11, p = 0.74) nor EL (F(1,97) = 0, p = 0.96) was observed for the SD group. Observed error rates for both control (50.34%, HEL; 9.0%, LEL) and SD (49.63%, HEL; 6.92%, LEL) groups were consistent with target rates, and while not significant (F(1,95) = 3.1, p = 0.081), a trend toward lower error rates in SDs was observed.

At first glance, it may seem puzzling that the task manipulations did not lead to effects on RT or error rate. In fact this is consistent with earlier studies of the change signal task, where RT effects and error rate effects were either absent or very minimal (Brown and Braver, 2005, 2007). This is due to the fact that the change signal task requires subjects to respond in a very narrow time window, so speeding up or slowing down will lead to greater errors. This works against differences in RT. Likewise, the change signal task strongly controls error rates. If subjects increase their accuracy, then the change signal task will increase the change signal delay, which makes it more difficult for subjects to respond correctly. In this way, the change signal task controls error rates to the specific target error rate, and so differences across conditions are not expected.

3.2. fMRI results

In Fig. 1B, we show effects of RM and EL predicted by the PRO model for our two hypotheses that SDs are more sensitive to reward (bottom panels) or that SDs are more risk-seeking (top panels). Briefly, the PRO model suggests that if SDs are more reward sensitive than non-substance dependent controls, SDs should show increased effects of reward magnitude and decreased effects of error likelihood relative to controls. In contrast, if increased risk-seeking underlies ACC activity in SDs, effects of both reward magnitude and error likelihood should increase relative to non-substance dependent individuals. In order to test the hypothesis that enhanced reward salience underlies differences in ACC activity in SD's as compared to non-substance dependent controls, we looked first at the between-groups differences in the error likelihood effect ((Control/HEL − Control/LEL) − (SD/HEL − SD/LEL)). Because the PRO model makes strong a priori predictions regarding activity specifically within ACC, we restrict this contrast to an anatomically defined ROI for Brodmann areas 24 & 32 (WFUPickAtlas, dilation = 2). Consistent with the reward salience hypothesis, and inconsistent with the risk-seeking hypothesis, the contrast identifies a region within ACC (Fig. 3A) showing greater effects of error likelihood in non-SDs versus SDs (peak voxel, MNI: −2, 0, 50, cluster level p < 0.001, k = 377 voxels/1336.4 mm³). Specifically, Fig. 3C shows that in this region, controls show reduced activation in the low error likelihood condition, for both high and low reward magnitude conditions. Overall activity in this region was greater in the SDs. Next, in order to test the prediction that effects of reward magnitude will be enhanced for SDs vs. Controls, we performed an ROI analysis using the region identified in our previous analysis using the contrast ((SD/HRM − SD/LRM) − (Control/HRM − Control/LRM)). Because error likelihood and reward magnitude are orthogonal in our design, this analysis is unbiased. This analysis revealed no significantly different voxels within the region. Instead, the inverse contrast ((Control/HRM − Control/LRM) − (SD/HRM − SD/LRM)) identified a significant cluster of voxels (Fig. 3B; peak voxel, MNI: 0, 2, 46, cluster level p = 0.023, k = 67 voxels/237.5 mm³), suggesting that reward magnitude effects within this region are attenuated for SDs. Interestingly, this pattern of results, (Fig. 3C) is consistent with the model-derived hypothesis that increased risk-aversion underlies differences in ACC activity for SDs. Specifically, the model predicts (Fig. 1B, upper right frame) that control subjects should show both enhanced RM and EL effects in ACC relative to SDs only if the degree of risk aversion for SDs is higher. This qualitative pattern is indeed observed in our sample (Fig. 3D).

Fig. 3.

A) Within ACC, greater error likelihood effects were observed for the non-substance dependent (non-SD) compared to the substance dependent (SD) group, consistent with the hypothesis that SD individuals are more sensitive to reward and inconsistent with the hypothesis that SD participants exhibit increased risk-seeking. B) Within the region showing increased EL effects for controls, a cluster was identified showing enhanced RM effects for non-SD versus SD groups, inconsistent with both the reward sensitivity and risk-seeking hypotheses. C) Percent signal change plotted for non-SD and SD individuals for the 4 EL/RM conditions for Go/Correct trials only. D) RM and EL effects for SDs, controls, and the between-groups differences. Both RM and EL effects are larger for controls than for SDs, consistent with model predictions suggesting that SDs are more risk averse than non-substance dependent individuals (cf. Fig. 1B).

We additionally carried out the analyses described immediately above using a more conventional, whole-brain approach with a cluster threshold of 0.001. While the results of this set of analyses were attenuated due to the more stringent threshold, they nevertheless reproduced the salient results. The contrast investigating between-groups differences in EL effects yielded a significant cluster with a peak activation slightly dorsal and lateral to the peak reported in the anatomically-constrained analysis (peak voxel, MNI: −2, −2, 56, cluster level p = 0.007, k = 101 voxels). Using this cluster as an ROI, we then investigated group differences in RM effects, identifying a single significant voxel that nevertheless passed cluster level correction (MNI: 0, 0, 46, cluster level p = 0.049, k = 1 voxel).

In order to assess whether differences in ACC activity between the two groups were driven primarily by differences in reward salience or differences in risk attitudes, we fit PRO model activity for each subject to the percent signal change (Fig. 4) in the region identified using the between-groups EL contrast. At the group level, parameter estimates for both reward/omission salience (λ) and risk attitudes (γ) were significantly different between SDs and non-SDs, indicating that SDs are both more sensitive to reward (higher mean estimated λ; t(47) = 3.54, p < 0.001) and more risk-averse (higher mean estimated γ; t(47) = 3.76, p < 0.001) as compared to controls. There were no significant differences between groups for the scaling parameter (t(47) = .299, p = 0.766). Across all subjects, parameter estimates for risk-aversion and reward salience were significantly, positively correlated (Fig. 4; r = 0.556, p < 0.001), indicating that increases in sensitivity to reward were associated with increased risk aversion. The scaling parameter was not correlated with either risk aversion (r = 0.06, p = 0.682) or reward salience (r = 0.180, p = 0.215) across all subjects.

Fig. 4.

Best fit parameters for the PRO model fit to individual data. Parameters found for SDs (open circles) were significantly more reward sensitive as well as significantly more risk-averse than parameters obtained for controls (crosses). Across all subjects, aversion to risk was positively correlated with reward sensitivity.

In order to test our assumptions of risk-neutrality and reward/omission indifference in our control group, we conducted t-tests between the parameters estimated for our control subjects for reward/omission salience (λ) and risk attitudes (γ) and the values used to derive our a priori model predictions (Fig. 1B). We find that both assumptions are violated. Estimated reward/omission salience parameters were significantly greater than 0.5 (t(23) = 3.133, p = 0.0047), indicating that control subjects were more sensitive to reward omission than reward. Estimated risk attitude preferences were significantly greater than 0 (t(23) = 2.507, p = 0.0197), indicating that control subjects were risk averse. Although the assumptions of our a priori model simulations were violated, this does not necessarily indicate that the predictions themselves were invalid. The direction of the predicted EL and RM effects for the various hypotheses outlined in Fig. 1B is relative to the control group, and although the magnitude of the between groups differences in effects may change depending on the particular risk preferences and reward sensitivity of the control group, the direction of those effects is preserved.

Fits of the model to fMRI data reveal substantial heterogeneity in SDs, suggesting that differences in RM and EL effects between SDs and the control group may be driven primarily by one of the SD subgroups (see Table 1). We investigated this possibility by conducting one-way ANOVAs for the RM and EL effects in SDs only, with the 3 subgroups as the main factor. These analyses were conducted using the ROI identified by the Control–SD contrast for effects of error likelihood, as described above, and the threshold for significance was set at p = 0.01 uncorrected. No effect of group was observed for the EL ANOVA. However, for the RM analysis, a single voxel was identified (MNI: −6, 8, 42, p = 0.007 uncorrected). Pairwise t-tests between groups showed that this group difference was driven primarily by the alcohol-only subgroup. For both the comparison between the alcohol-only dependent group vs. the marijuana-only group, a cluster of voxels (peak voxel MNI: −2, 2, 42, p = 0.004 uncorrected, cluster extent = 8; cluster significance p = 0.217 correct) was observed. A similar cluster was observed for the comparison between the alcohol-only group and the polydrug-dependent group (peak voxel MNI: −6, 8 ,42; p = 0.001 uncorrected; cluster extent = 14; cluster significance p = 0.156 corrected). Neither cluster passed significance, rendering these findings tenuous at best. Due to the nature of the process used to recruit substance-dependent subjects, this study is inadequate to investigate differences in activity related to choice of substance.

Because our control group was selected randomly from the population without screening for substance dependence, it is possible that the between-groups differences observed in our model-fit parameters are driven, in part, by undiagnosed substance-dependent individuals. In order to assess this possibility, we conducted a permutation test in which individuals were excluded from the control group. Note that this is a fairly conservative test in that we assumed any individual may be substance dependent regardless of the value of the dependent variables (model parameters) that are assumed to be related to substance dependence. The permutation test yielded a probability of rejecting the null hypothesis of no between-group differences of 0.9994 for the risk parameter γ, and 0.9915 for the reward salience parameter λ. In other words, if we had screened out undiagnosed substance-dependent individuals from the control group, the probability that we would no longer find the observed effects is p < 0.001 for γ, and p < 0.01 for parameter λ. This suggests that contamination of the control group is not responsible for the differences observed in our model parameters.

3.3. fMRI and behavior relationships

The above results show how participants differ in the brain activity parameters recovered from the model, but the next question is how these differences relate to behavior. The change signal task does not directly measure participants' decision preferences or corresponding behavioral utility function. To address this question, we analyzed data from the DOSPERT (Weber et al., 2002). We specifically looked at self-reported risk-taking likelihood for gambling (as reported in our earlier work (Brown and Braver, 2007)), and also health and safety risk-taking likelihood, which includes substance use. With the DOSPERT, we found that relative to controls, SDs were more likely to gamble (p < 0.04) and take health and safety risks (p < 0.005). This is consistent with the reduced reward omission parameters derived from the ACC model fits in SDs relative to controls.

With regard to risk aversion, we further correlated the DOSPERT health and safety risk-taking likelihood across all subjects with the model risk aversion parameter derived from the ACC, and we found a trending positive correlation (r = .22, p = 0.07). While just shy of statistical significance, this is consistent with greater risk aversion model parameters in those more likely to use substances. It is important to note here that risky behavior and risk aversion entail different concepts, despite the shared use of the word “risk” (Krawitz et al., 2010). The model risk aversion parameter refers specifically to the concavity of the utility function used in model decision-making, with greater risk aversion corresponding to less willingness to pursue higher-valued but less certain options. This concept of risk-taking behavior involves decisions that may lead to negative consequences. Indeed, previous work has noted that individuals whose estimated utility function has a high degree of concavity (where concavity is synonymous with aversion to risk in the judgment and decision-making literature) may engage in impulsive behaviors relative to individuals with less concave utility functions (Anderhub, Güth, Gneezy and Sonsino, 2001; Andersen, Harrison, Lau and Rutström, 2008; Apesteguia and Ballester, 2014; Keren and Roelofsma, 1995; Pine et al., 2009). Thus, it is not a contradiction to note that higher risk aversion (utility function concavity) correlates with higher risk taking behaviors (decisions with greater potentially negative consequences). This can be the case if higher utility function concavity means that smaller payouts are valued more highly, so participants would be more willing to pursue them relative to controls.

4. Discussion

In this study, we tested two hypotheses, derived from a computational model of ACC, regarding differences in ACC activity between substance-dependent (SD) and non-substance dependent (non-SD) users. Using classical fMRI analyses, we found a region in dACC showing that effects of both reward magnitude and error likelihood are attenuated in SDs relative to the non-SD individuals. Somewhat surprisingly, this finding suggests that differences in ACC activity in SDs could be attributed to increased risk aversion relative to the non-SD individuals. In a second set of model-based analyses, we fit activity from a computational model of ACC to fMRI data from individual subjects, in order to estimate parameters indicating reward salience and risk attitudes. In these analyses, we find that ACC activity in SDs is best explained as a combination of both increased reward salience, as well as increased risk-aversion.

Characterizing SDs as being risk-averse, as opposed to risk-seeking, seems quite paradoxical considering that substance use itself is generally considered a risky behavior. However, the concept of risk aversion as we use it in this study is common in the judgment and decision-making literature (Tversky and Kahneman, 1992). The degree to which an individual is risk-averse is reflected by how quickly a utility function, representing the subjective value of a monetary reward, saturates; the utility function for highly risk-averse individuals rapidly approaches an upper bound for relatively small rewards. Fig. 2 shows that the “risk averse” (concave value function) subjects actually over-weight smaller rewards. In other words, even small rewards are more salient and valuable to the “risk averse” subjects than to the subjects with a more convex value function. By implication, in “risk averse” subjects, larger rewards carry less additional utility relative to small rewards. In this way, those who show a concave value function may be more inclined to value and pursue even small rewards such as immediately available drugs, as larger rewards may not be deemed worth the additional effort that may be required. This yields a consistent account of SD behavior. The over-weighting of smaller rewards in SDs also leads to a novel prediction: those with more concave value functions should also show greater sensitivity to reward. Fig. 4 shows that the SDs do show both greater sensitivity to reward and greater value function concavity (risk aversion), and the two are positively correlated. In this way, the model provides a coherent account of SD behavior, with consistent empirical results, in terms of over-weighting of small reward values.

Previous literature is consistent with these findings and provides a delineation of different kinds of risk. In agreement with our findings, a previous study investigating how SD value monetary rewards finds that substance users weigh intermediate and large rewards approximately equally (Goldstein et al., 2007a), consistent with a rapidly saturating utility function. The definition of risk as “utility function concavity” differs from the health psychology risk defined as “decisions that may lead to harmful outcomes”, or even from the finance definition of risk as “greater variance in the outcome distribution” (Krawitz et al., 2010). These concepts from the existing literature must be distinguished carefully to avoid confusion and seemingly contradictory uses of the word risk. Here, most SDs were more sensitive to reward than reward omission (i.e. greater reward-seeking and reduced sensitivity to reward omission); they showed more risky behavior (in the health sense of risk, i.e. using alcohol and drugs), and they showed more risk aversion in the sense of utility function concavity. Our results do not directly address risk preferences in the sense of preferring greater or lesser variance in the outcome distribution. It is possible for SDs to be less risk averse in the sense of a less concave utility function, while also exhibiting more reward-seeking and decreased sensitivity to reward omission. These consist of the SDs plotted in the upper left quadrant of Fig. 4. Still, SDs as a whole showed more risk aversion (utility function concavity) relative to controls. Overall, these results and the general increase in activity in SDs shown in Fig. 3 are consistent with the reward salience hypothesis as discussed in the Introduction section, i.e. that substance dependence is not merely a product of risky behavior but rather depends on specific alterations in reward valuation.

How does increased risk-aversion contribute to the qualitative differences observed between SD and non-SD individuals in this study? The PRO model learns predictions of likely response–outcome conjunctions, regardless of affective valence. However, this does not preclude the possibility that affectively valenced events may bias predictions learned by the PRO model due to relative differences in the salience of affectively positive and negative events. In the ICST, four RO conjunctions were possible (Go/Correct, Go/Error, Change/Correct, and Change/Error), and the relative salience of each of these outcomes was modeled as the utility of the total number of points received (in the case of correct outcomes), or the difference in utility between the total number of points possible on a trial and the actual points received (for error trials). In the case of High Reward Magnitude trials, subjects received points even for incorrect responses. In the case of a highly risk-averse subject, the subjective utility of receiving 1000 points is approximately the same as the utility of receiving 2000 points. Consequently, when a risk-averse subject commits an error on an HRM trial, the salience of that event is negligible, leading to decreased predictive activity on HRM trials, especially in the HEL condition in which errors are frequent.

Perhaps less controversial is our finding that, for SDs, the pattern of activity we observed in ACC reflects, in part, increased sensitivity to rewards versus reward omission, while the reverse is true for non-SD individuals. This is consistent with prior findings of reduced punishment sensitivity in SDs at the neural (Brown and Braver, 2008; Fukunaga et al., 2013) and behavioral levels (Yechiam et al., 2005). Our findings of increased reward omission salience in the non-SD group are, to the extent that the omission of a reward can be considered a loss, consistent with the well-known phenomenon that “losses loom larger than gains” (Kahneman and Tversky, 1979), and further suggest that inattention to potential aversive outcomes may contribute to maladaptive decision making in SD individuals. Notably, our classical analyses failed to find evidence of increased activity in ACC associated with increased reward magnitude in the SD group as predicted by the PRO model. We attribute this to the interaction of risk-aversion and reward salience; parameter estimates for reward salience and risk-aversion from our model-based analysis were strongly correlated across all subjects, as well as within each group, suggesting that a common mechanism may underlie both increased risk-aversion and enhanced reward salience.

4.1. Limitations and future questions

One of the limitations of our study concerns participant recruitment, which may have led us to slightly underestimate the magnitude of the effects reported above. Specifically, our control participants were recruited from the general population for an earlier study (Alexander and Brown, 2010). They were not explicitly screened to rule out those with substance dependence, so a small percentage may have substance use disorders. If we had screened them out, then the control and SD participants may have been even more different from each other, which would likely have led to larger effect sizes in the effects we observed above. Nevertheless, we showed that the SD participants were significantly more likely to smoke and drink heavily relative to controls. Also, we have shown with a permutation analysis that the between group differences we observed would be unlikely to change if we had a purely non-SD control group (p < 0.01).

We also acknowledge that our findings may be limited by the heterogeneous nature of our mixed SD group, as these participants are known to use various types of drugs. On the other hand, our findings are in line with current movements to conceive of diagnoses on more of a dimensional scale, where particular domains and constructs are not definitive (e.g., withdrawal versus no withdrawal). Although our findings suggest that activity in ACC reflects an interaction between reward salience and risk aversion, it remains an open question as to whether ACC itself is responsible for biasing processing of information related to reward delivery versus omission, or whether such biases are introduced prior to cingulate involvement. SDs exhibit bias both in attention and decision-making (Field and Cox, 2008; Hester et al., 2006; Hester et al., 2009). In the PRO model, ACC is regarded as a region involved in learning predictions of likely outcomes regardless of affectively valenced events such as gain or loss. However, this does not preclude the possibility that inputs to ACC may themselves be biased based on affective information in a way that varies with individual differences (Brown and Braver, 2008). In either case, the ACC may impose an attentional bias, as the rapidly saturating utility function and greater reward sensitivity in SDs may combine to increase the salience of rewarding cues, especially drugs.

The dopaminergic system, long associated with reinforcement and reward (Schultz et al., 2000), is a likely source of biases related to reward salience. Input to ACC from midbrain dopamine (DA) neurons reflecting the value of an event may modulate value-free input to ACC coding for the objective category of an event, thus weighting predictions acquired by ACC regarding likely outcomes. Similarly, a promising candidate as a source for biases related to risk-aversion is the serotonergic system. ACC is heavily innervated by neurons in dorsal raphe (Lidov et al., 1980), and, computationally, a number of potential roles for serotonin (5-HT) related to judgment and decision making have been suggested. It has been variously proposed that 5-HT reflects impulsivity (Daw and Doya, 2006), temporal discounting (Doya, 2007), prediction of long-term punishment (Daw et al., 2002), as well as aversion to risk (Krichmar, 2008). Furthermore, the DA and 5-HT systems are thought to interact with one another, both at a computational level (Daw et al., 2002), as well as anatomically through overlapping projection targets (Brown and Molliver, 2000; Lidov et al., 1980; Oades and Halliday, 1987), consistent with our finding of a correlation between reward salience and risk-aversion parameters.

More generally, this study contributes to the growing body of literature in the area of computational psychiatry (Maia and Frank, 2011; Montague et al., 2012). In brief, the goal of computational psychiatry is to apply quantitative, model-based approaches to identify psychological or neural variables that contribute to psychopathology and behavioral disorders. In this respect, our study offers a novel contribution to this effort through associating ACC activity in SD individuals with enhanced reward processing/attenuated processing of reward omission and increased risk aversion. Although it has previously been hypothesized that increased reward salience, potentially mediated by DA, might underlie substance dependence, to the best of our knowledge this is the first work linking substance use with risk aversion. While additional studies are needed to confirm this link, this study supports the growing movement to apply computational modeling as a tool for making differential diagnoses and subgroup detection characterized by varying disease processes (Stephan and Mathys, 2014), thus moving from a symptom-based to a more mechanism-based disorder classification underlying specific dysfunctions of neurocognitive systems (Goschke, 2014).