Belief about nicotine selectively modulates value and reward prediction error signals in smokers

Significance

Nicotine is the primary addictive substance in tobacco, which stimulates neural pathways mediating reward processing. However, pure biochemical explanations are not sufficient to account for the difficulty in quitting and remaining smoke-free among smokers, and in fact cognitive factors are now considered to contribute critically to addiction. Using model-based functional neuroimaging, we show that smokers’ prior beliefs about nicotine specifically impact learning signals defined by principled computational models of mesolimbic dopamine systems. We further demonstrate that these specific changes in neural signaling are accompanied by measurable changes in smokers’ choice behavior. Our findings suggest that subjective beliefs can override the physical presence of a powerful drug like nicotine by modulating learning signals processed in the brain’s reward system.

Abstract

Little is known about how prior beliefs impact biophysically described processes in the presence of neuroactive drugs, which presents a profound challenge to the understanding of the mechanisms and treatments of addiction. We engineered smokers’ prior beliefs about the presence of nicotine in a cigarette smoked before a functional magnetic resonance imaging session where subjects carried out a sequential choice task. Using a model-based approach, we show that smokers’ beliefs about nicotine specifically modulated learning signals (value and reward prediction error) defined by a computational model of mesolimbic dopamine systems. Belief of “no nicotine in cigarette” (compared with “nicotine in cigarette”) strongly diminished neural responses in the striatum to value and reward prediction errors and reduced the impact of both on smokers’ choices. These effects of belief could not be explained by global changes in visual attention and were specific to value and reward prediction errors. Thus, by modulating the expression of computationally explicit signals important for valuation and choice, beliefs can override the physical presence of a potent neuroactive compound like nicotine. These selective effects of belief demonstrate that belief can modulate model-based parameters important for learning. The implications of these findings may be far ranging because belief-dependent effects on learning signals could impact a host of other behaviors in addiction as well as in other mental health problems.

One materialist view of mental function suggests that even the most abstract beliefs can be represented in terms of physiological states available to the brain (1). These mappings are critical in conditions like drug addiction, where our ignorance of how prior beliefs about drugs influence physiological processes related to drugs of abuse presents a profound challenge to the understanding of the mechanism and treatment of addiction (2, 3). Although extensive work has shown that addictive drugs act on the mesolimbic dopaminergic (DA) pathway (2, 4), it has become clear that these purely biochemical explanations are not sufficient to account for the huge heterogeneity among drug-dependent individuals and the low success rate of quitting and remaining drug-free (2, 5) and that cognitive factors such as beliefs and expectations have a profound impact on drug-related neurobiological effects (6, 7).

Beliefs are known to contribute to the placebo effect. The placebo effect is a treatment effect not caused by the physical presence of an active drug, but rather by the meaning ascribed to it and the subjective expectation of receiving a treatment (8, 9). A subject's belief that he or she is receiving a treatment could lead to observable improvement even in the absence of active drugs. These treatment effects are putatively accomplished by neurobiological processes usually associated with pharmacological actions of active drugs, even though active drugs are not administered (10–14). Interestingly, beliefs also directly impact behavioral (15–20) and neurophysiological (21–26) responses when addictive drugs are administered. Drug dependence is a learned process in which cognitive factors are critical (2, 5, 27–30).

Thus, uncovering the mechanisms by which belief modifies drug responses is crucial for understanding the causes of and the treatments for addiction. Importantly, fine-grained quantitative analyses based on learning models is required to gauge the impact of abstract beliefs on computationally explicit signals and could help lead to mechanistic explanations for the role of beliefs in drug addiction. In this study we used computational modeling and model-based functional magnetic resonance imaging (fMRI) in chronic smokers to investigate the impact of beliefs on computational and neural learning signals.

We hypothesized that smokers’ prior beliefs about nicotine would influence nicotine effects by selectively modulating neural learning signals and subsequently modifying choice behavior. Beliefs about neuroactive substances such as alcohol (21, 22, 26) and cocaine (23, 24) have been shown to affect brain activity. Striatal DA levels are also modulated by prediction errors about alcohol (21). It is thus reasonable to suspect that beliefs about nicotine could modulate the key neural signals involved in nicotine addiction (4, 31). These same neural signals that guide learning also guide choice behavior (32, 33) and are disrupted in nicotine addiction (4, 31, 34–37). Thus, factors that might modulate neural signals in the striatum, such as smokers' prior beliefs, could also impact reinforcement learning behavior in smokers.

To test this hypothesis, we used a within-subject balanced placebo design where we manipulated 24 smokers’ beliefs about the absence or presence of nicotine (belief: told “no nicotine” vs. told “nicotine”) in a denicotinized cigarette or a cigarette with nicotine (cigarette: placebo vs. nicotine) smoked immediately before an fMRI session in four separate visits (Fig. 1 and Materials and Methods). In each visit, subjects carry out a sequential investment task where they place a bet $b_t$ at time t, experience a fractional change in market price $r_t=(p_t-p_{t-1})/p_{t-1}$ ($p_t$ is the market price at time t), and obtain a gain (or loss) $g_t=b_t{r}_t$. Therefore, the task possesses two computationally explicit variables important for learning: (i) a passive value variable market return $r_t$ that does not depend on choice behavior ($b_t$ independent) and (ii) a choice-dependent reward prediction error variable $T{D}_t$ defined as the actual gain minus the expected gain $\widetilde{g_t}-\widetilde{b_t}$ (∼ means z scored), where $\widetilde{b_t}$ is taken as the proxy for expected return (37–39) (Materials and Methods). Both variables have been connected to computational models of dopaminergic function (40). In our previous work using similar paradigms, reward prediction error shows up as a strong neural signal in the striatum (37–39); the neural correlates of the value variable $r_t$ have not been reported using the same task.

Fig. 1.

Experimental procedure. Twenty-four smokers completed four sessions of fMRI scanning in four separate visits. In each session, subjects were given a denicotinized cigarette or a cigarette with nicotine (cigarettes: placebo vs. nicotine) to smoke and were told that the cigarette had no nicotine or had nicotine (belief: told no nicotine vs. told nicotine). Immediately after smoking, subjects performed a sequential investment task in the scanner where they made 20 investment decisions $b_t$ (0 ∼ 100% of current running total, number displayed on lower left side of the screen, e.g., $104) during each market, for a total of 10 markets. Market return $r_t$ was displayed on the lower right side of the screen (e.g., −1.2%). Carbon monoxide (CO) levels were measured both in the beginning and at the end of the experiment (Materials and Methods and Fig. S2).

Results

Impact of Belief on Neural Responses to Value ${\mathit{r}}_{\mathit{t}}$.

We first examined the impact of belief on neural activities related to the value signal $r_t$ (Fig. 2 A and B). When subjects were told “nicotine in cigarette” and smoked nicotine, they showed significant activations in bilateral ventral striatum related to the value signal $r_t$ (Fig. 2A and Table S1, P < 0.05 corrected for family-wise error, P_FWE; Materials and Methods). Strikingly, these $r_t$-related striatum activations were significantly attenuated when smokers were told “no nicotine in cigarette” and had nicotine (Fig. 2A and Table S1, P_FWE < 0.05).

Fig. 2.

Impact of belief on the value signal $r_t$. (A) Beliefs about nicotine modulated $r_t$-related ventral striatum activation (P_FWE < 0.05; displayed at P < 0.001 uncorrected for visualization). (B) Region of interest analysis (peaks [−12, 8, −6] and [12, 10, −6]; Table S2) (39) confirmed whole-brain results shown in A. (C) The weight of market return $r_t$ on choice behavior (next bet $b_{t+1}$) was significantly reduced when told no nicotine than told nicotine, despite the presence of nicotine in both conditions. (D) Bayesian analysis confirmed the separation between the posterior distributions of the behavioral regression coefficient of market return $r_t$ of told nicotine and told no nicotine. ***P < 0.001. Data are represented as mean ± SEM.

We further extracted parameter estimates related to $r_t$ from regions of interest (ROIs) defined by $r_t$-related striatal activations from an independent dataset, using the same paradigm (39) (Materials and Methods and Table S2). This analysis confirmed that the belief of no nicotine significantly decreased bilateral striatal activity in the presence of nicotine relative to the belief of “nicotine present” in these smokers (Fig. 2B, paired t test, t₍₂₃₎ = 2.62 and P < 0.05 for parameter estimates averaged over bilateral striatum). These results demonstrate a profound impact of belief about nicotine on neural activities in the ventral striatum when subjects smoked nicotine.

Impact of Belief on ${\mathit{r}}_{\mathit{t}}$-Driven Choice Behavior.

We then examined whether such neural changes are accompanied by explicit behavioral alternations modulated by belief (Fig. 2 C and D). We calculated the weight of market return $r_t$ on next bet $b_{t+1}$, using a linear mixed-effects multiple-regression model (details in Materials and Methods). Compared with when told nicotine, the regression coefficient of $r_t$ on next bet $b_{t+1}$ was significantly reduced when subjects were told no nicotine [t_(16,885) = 5.07, P < 0.0001, Fig. 2C; Table S3 shows list of all regressors], even though they smoked nicotine in both conditions.

An additional Bayesian analysis confirmed such separation between the regression coefficient distributions of told no nicotine (mean = 2.75) and told nicotine (mean = 3.82) conditions, when nicotine was delivered (Fig. 2D, posterior probability of told nicotine > told no nicotine = 1; Materials and Methods). Both analyses suggest that belief about the absence of nicotine significantly tempered the impact of the value signal on choice behavior, compared with when subjects believed there was nicotine in the cigarette, despite the presence of nicotine in both conditions.

Impact of Belief Neural Responses to Reward Prediction Error $\mathit{T}{\mathit{D}}_{\mathit{t}}$.

Next, we examined the neural impact of belief on the reward prediction error signal $T{D}_t$ when subjects smoked a cigarette with nicotine (Fig. 3 A and B). Whole-brain analysis suggests that there were significant activations in the striatum related to the reward prediction error signal $T{D}_t$ when subjects were told nicotine and smoked nicotine (Fig. 3A and Table S4; P_FWE < 0.05). Similar to the value signal $r_t$, these $T{D}_t$-related striatum activations were attenuated when smokers were told no nicotine in cigarette (Fig. 3A and Table S4; P_FWE < 0.05), even though nicotine was delivered in both conditions.

Fig. 3.

Impact of belief on the reward prediction error signal $T{D}_t$. (A) Beliefs about nicotine modulated $T{D}_t$-related striatum activation (P_FWE < 0.05; displayed at P < 0.001 uncorrected for visualization). (B) Region of interest analysis (peaks [−18, 0, −8] and [12, 8, −2]; Table S2) (39) confirmed whole-brain results shown in A. (C) The weight of reward prediction error $T{D}_t$ on choice behavior (next bet $b_{t+1}$) was significantly reduced when told no nicotine than told nicotine, in the presence of nicotine. (D) Bayesian analysis confirmed the separation between the posterior distributions of the behavioral regression coefficient of reward prediction error $T{D}_t$ of told nicotine and told no nicotine. ***P < 0.001. Data are represented as mean ± SEM.

These whole-brain analysis results were further confirmed by an ROI analysis based on $T{D}_t$-related striatum activation from an independent study, using the same paradigm (39) (Table S2): There was significant attenuation in neural responses in the striatum to $T{D}_t$ when smokers were told no nicotine relative to when they were told nicotine, although they smoked nicotine in both conditions [Fig. 3B and paired t test, t₍₂₃₎ = 1.72, P = 0.1 for left striatum ROI; not significant for right striatum ROI].

Impact of Belief on $\mathit{T}{\mathit{D}}_{\mathit{t}}$-Driven Choice Behavior.

We then examined whether such neural changes are reflected in behavior related to the reward prediction error signal $T{D}_t$. Using a second linear mixed-effect regression model (Materials and Methods), we found that there was a significant reduction in the influence of $T{D}_t$ on the next investment $b_{t+1}$ in the told no nicotine condition compared with the told nicotine condition, despite the presence of nicotine in both conditions [Fig. 3C and Table S5; t_(16,885) = 3.72, P < 0.001].

A Bayesian analysis further confirmed the separation between the distributions of these regression coefficients of told nicotine (mean = 0.22) and told no nicotine (mean = 0.16) conditions when nicotine was delivered (Fig. 3D, posterior probability of told nicotine > told no nicotine = 0.996; Materials and Methods). Taken together, these results suggest that belief about nicotine also modulated the neural representations of the reward prediction error signal $T{D}_t$ and brought related behavioral changes in smokers.

Belief Did Not Modulate Neural Activities in Visual Attentional Areas.

The next question we asked was whether the effect of belief was specific to learning signals or related to global changes in visual attention. To answer this question, we examined neural activities related to viewing market price (onset of market reveals rounds 2–19; details in Materials and Methods). The whole-brain analysis of the “market price reveal” regressor showed similar levels of activation in inferior and middle occipital gyri, inferior parietal lobule, and inferior frontal gyrus when subjects were told there was nicotine and smoked nicotine and when they were told no nicotine and smoked nicotine (Fig. 4A and Table S6, P_FWE < 0.05). Direct comparison between told no nicotine and told nicotine conditions did not yield any significant activation even at a very liberal threshold of P < 0.01 uncorrected. Furthermore, ROI analysis based on visual attentional seeds from the same independent study (Fig. 4B and Table S7) confirmed that there was no significant difference in any of these visual attention-related ROIs (inferior occipital gyrus, inferior parietal lobule, and inferior frontal gyrus) between told no nicotine and told nicotine conditions (Ps > 0.1). Taken together, these results suggest that belief did not impact neural activities in visual attentional regions and that the effect of belief was selective to the value signal and the reward prediction error signal.

Fig. 4.

Belief did not modulate neural activities in visual attentional regions. (A) Whole-brain analysis of the market reveal regressor suggests that smokers showed equivalent levels of activations in occipito-temporal visual areas and fronto-parietal regions involved in attention when told no nicotine and told nicotine (P_FWE < 0.05, displayed at P < 0.001 uncorrected for visualization, Table S6; no significant activation in contrast image even at P < 0.01 uncorrected). (B) ROI analysis based on an independent dataset (39) confirmed that belief about nicotine did not significantly modulate neural activities in temporal, parietal, and frontal ROIs: inferior temporal gyrus [48, −62, −2], inferior parietal lobule [30, −54, 48], and inferior frontal gyrus [52, 12, 20] (all Ps > 0.1; details in Materials and Methods and Table S7).

Impact of Belief on the Value Signal and the Reward Prediction Error Signal in Placebo Conditions.

The above results demonstrate the effects of belief about nicotine on learning signals when subjects smoked a cigarette that contained nicotine. We also examined the impact of belief on the value signal $r_t$ and the reward prediction error signal $T{D}_t$ in the placebo conditions (Fig. S1 and Tables S1 and S3–S5). Belief of no nicotine did not significantly modify neural activations in the striatum related to $r_t$ (Fig. S1A), yet reduced $T{D}_t$-related right striatum activation (Fig. S1B) in the placebo conditions. Behaviorally, belief about the absence of nicotine reduced the weights of both the value signal $r_t$ (P < 0.05; Fig. S1C) and the reward prediction error signal $T{D}_t$ (P < 0.05; Fig. S1D) on choice behavior when subjects smoked a denicotinized cigarette. These results further demonstrate the impact of belief on computational learning signals even in the absence of nicotine and suggest that such impact had differential effects on the value signal $r_t$ and the reward prediction error signal $T{D}_t$ neurally.

Discussion

Our main findings are twofold. First, smokers’ beliefs about nicotine’s presence modulated the neural representation of a computationally explicit value signal $r_t$ in the striatum, as well as the impact of $r_t$ on choice behavior. Second, belief about nicotine also modulated neural activity in the striatum related to the reward prediction error signal $T{D}_t$ and the impact of $T{D}_t$ on subjects’ choice behavior. We further demonstrated that these effects of belief were not observed for neural activities related to general visual attentional processes in fronto-parietal and occipito-temporal regions and were specific to striatal learning signals. These results provide compelling evidence demonstrating that prior beliefs about nicotine have the capacity to override the presence of a powerful neuroactive drug like nicotine by selectively modulating biophysically described processes in a fashion that correlates with measurable impact on learning and choice behavior.

Several previous studies have examined neural responses modulated by beliefs and expectancy about substances of abuse such as cocaine and alcohol (21, 23, 24, 26). In cocaine abusers, expectations about receiving methylphenidate increased metabolism measured by positron emission tomography (PET) in the thalamus, occipital lobe, and cerebellum, but decreased metabolism in the orbitofrontal gyrus (24). Expectations of receiving cocaine (23) and alcohol (26) were found to modulate fMRI blood-oxygen-level dependent (BOLD) signals in frontal and anterior cingulate regions in addicted individuals. Interestingly, one PET study found that negative drug prediction errors (i.e., alcohol expected but not delivered) decreased striatal DA concentrations while positive drug prediction errors (i.e., alcohol not expected but delivered) increased striatal DA levels in individuals with alcohol dependence (21), which is consistent with our finding of modifiable striatal responses by beliefs. Furthermore, our findings extend previous results showing the impact of beliefs on behavioral performances and subjective states related to nicotine intake (15–17, 20) by detailing the specific neural and computational mechanisms underlying the impact of beliefs on drug effects.

These selective effects demonstrate that belief can modulate model-based parameters important for learning and suggest that belief serves as an important cognitive mechanism in addiction. The implications of these findings are far ranging, because learning is critically involved in drug addiction (2, 5, 27–30). Addictive drugs bring rewarding feelings and subjective utility, which establishes the chosen drug as a reinforcer for drug-seeking and drug-taking behavior; over time, conditioned responses are developed and drug use evolves into a learned habit (5, 27–29). In this sense, learning is closely associated with motivational (“wanting”), affective (“liking”), and executive processes in addiction (29, 41). Thus, belief-dependent effects on generic learning signals could impact a vast range of other behaviors in addiction. Future studies using longitudinal designs are needed to address the issue of how belief directly influences learning and other aspects of addiction during the formation of addictive behavior, which would significantly advance our understanding of the mechanisms of the acquisition of addiction.

Our results also suggest the possibility of using manipulated beliefs to modify abnormal neural and behavioral responses in addicted individuals. To date, a great deal of effort has been made to identify methods that can reverse drug-related reward responses through manipulations of the mesolimbic DA pathway. In rodent models, pharmacological intervention (42, 43), genetic modification (34), and optogenetic stimulation (44) have shown initial success in reversing drug-induced behavior by acting on DA signaling. In humans, both pharmacological and cognitive therapies have been used, although studies have reported mixed findings regarding their general effectiveness (45). Here we show that the mere subjective belief of no nicotine in cigarette exerted a strong reversal effect by attenuating neural responses in the striatum, even in the presence of a powerful neuroactive drug like nicotine. Crucially, these neural effects brought measurable changes in smokers’ choice behavior. Taken together, these results suggest that cognitive beliefs could be as potent as pharmacological interventions in terms of modifying biophysical processes in the brain and changing behavior in addicted individuals. It remains an open question of whether systematically managed beliefs could reliably influence neural signals over time and result in long-lasting changes in behavior. Future longitudinal experiments could potentially answer this question, which will have important implications for the recovery from and treatment of addiction.

Furthermore, our results suggest that DA abnormality is critical, but not sufficient, to account for addiction. Addictive drugs and natural rewards have been shown to act on the same neurophysiological processes of DA signaling (28, 36). Increased neural activity in the striatum in response to drug reward and drug cues and the lack of striatum activation in response to other natural rewards in the absence of drugs have therefore been considered a hallmark of addiction (2, 4, 28, 29, 42). If drug-induced DA release were sufficient to account for addiction, however, nicotine intake would increase DA release and neural activities in the striatum regardless of contextual constraints or top–down beliefs about nicotine’s presence in our experimental setting. Here we show that although smokers did show a “normal” level of striatum activation related to reinforcement learning signals after nicotine intake, these neural activities can be significantly diminished by a top–down belief of no nicotine in cigarette even when the same amount of nicotine was administered. Thus, although it might account for physical dependence, DA abnormality alone is not sufficient to account for the whole collection of addictive symptoms. Cognitive mechanisms implemented in distributed neural circuitries beyond the mesolimbic DA system are also crucial for addiction (2, 3).

In summary, the current findings tap into an important yet underinvestigated question in addiction research of how high-level cognitive factors such as beliefs could impact or counter drug effects (30, 46). Knowing the exact mechanisms through which beliefs work would contribute to the understanding of the causes and treatments of addiction, as well as a wide range of other mental health problems. It would also lead to further inquiries about how top–down beliefs can modulate biological processes in general to effect personal and even societal changes.

Materials and Methods

Participants.

Twenty-eight smokers were recruited from community populations in Houston, Texas. Three smokers were excluded because they had carbon monoxide (CO) levels outside the acceptable range. One additional smoker was excluded because of obvious alcohol intoxication. Other exclusion criteria were (i) left handed; (ii) claustrophobia; (iii) Diagnostic and Statistical Manual of Mental Disorders, 4th edition, Axis I or II diagnosis (47), exclusive of nicotine dependence; (iv) pregnant; (v) contraindications to MRI, pacemaker, aneurysm clips, neurostimulators, cochlear implants, metal in eyes, steel worker, or other implants; (vi) active medical or neurologic disorder; (vii) history of drug dependence (other than tobacco or alcohol); (viii) history of head injuries resulting in loss of consciousness >10 min; and (ix) nonsmoker. This yielded a final sample of 24 smokers (9 females). Other demographic and clinical characteristics are as follows (mean ± SD): age, 39 ± 14 y old; education, 13 ± 3 y; daily cigarettes, 17 ± 9; smoking history, 22 ± 15 y.

Subjects had normal or adjusted to normal vision, had no contradiction to MRI, and reported no previous or current psychiatric or neurological conditions. Subjects were informed of the study requirements and provided written consent before participation. The study was approved by the Institutional Review Board of the Baylor College of Medicine ethics committee.

Experimental Protocol.

Smokers were instructed to smoke as usual and upon arrival at the laboratory, to smoke until satiated. Using a balanced placebo design, each smoker came to the laboratory on four separate days and received each of the four conditions: told no nicotine and received a cigarette with nicotine (Quest Brand, 0.6 mg of nicotine), told no nicotine and received a placebo cigarette (Quest Brand, 0.06 mg of nicotine), told nicotine and received a cigarette with nicotine, and told nicotine and received a placebo cigarette. Before the experiment, subjects were informed that they would receive either a nicotine cigarette or a placebo cigarette and that a research staff member would tell them whether they would be smoking a nicotine cigarette or a placebo cigarette. Therefore, subjects had no prior knowledge about the experimental manipulation and were debriefed only after they completed all four visits. The experiment was designed to be double blind. Although the experimenters administering the cigarette to the subject developed their own beliefs about the contents of the cigarettes for some fraction of the subjects, the double-blind protocol was not broken during data collection. Two other experimenters who were completely blind to the data collection procedure performed the data analysis.

The order of the four visits was randomly assigned to each subject. Upon arrival at the laboratory, exhaled CO was measured for each subject. CO level did not differ between told nicotine, smoked nicotine and told no nicotine, smoked nicotine conditions or between told nicotine, smoked placebo and told no nicotine, smoked placebo conditions before the experiment (Ps > 0.1; Fig. S2A). Subjects also completed a battery of surveys on demographic information and smoking history. All subjects then received a cigarette and suggestion immediately before the scanning session. CO levels were measured again after the scanning session: CO level did not differ between told nicotine, smoked nicotine and told no nicotine, smoked nicotine conditions or between told nicotine, smoked placebo and told no nicotine, smoked placebo conditions after the experiment (Ps > 0.1; Fig. S2B).

Stimuli and Task.

Participants performed a sequential investment task (Fig. 1) based on historical stock markets, similar to the ones used in previous studies (37–39). Participants were endowed with 100 monetary units at the beginning of the experiment and were informed that their final payment would be scaled according to their score in the experiment. Each subject played a total of 10 markets in each visit and each market block consisted of 20 event-related trials. At the beginning of each market block, a screen that indicated “new market” was shown, followed by a display of initial market history. The subject then made 20 sequential investment decisions b_t (0 ∼ 100% of current portfolio) without a time constraint, using a slider bar. Therefore, the trials were jittered by the natural variability in subjects’ response times. A total of 750 ms after they submitted their choices, the market price p_t was revealed and the fractional market price change and subjects’ portfolios were updated. Market information for all previous segments then remained on the screen until the end of each trial. The slider bar then changed from gray to red after an intertrial interval of 750 ms, and subjects started to make investment decisions for the next trial. A total of 40 different markets were used with 10 different markets for each of the four visits. The mean reaction time (RT) was 2.76 s with a SD of 1.27 s. The experimental length was 844 s on average with a SD of 253 s. There was no difference between conditions in terms of either RT or task length (all Ps > 0.1).

Behavioral Data Analysis.

Linear mixed-effects multiple-regression model.

We first examined the impact of the value signal market return r_t on subjects’ next bet $b_{t+1}$, using a linear mixed-effects multiple-regression model (48). This model allows flexible handling of complex group data structure and takes into account both fixed and random effects. The value signal at time t is defined as market return r_t, that is, the relative change in market price (p_t − p_t−1)/p_t−1. The regression was performed simultaneously across all four conditions by coding the four conditions (told no nicotine and received placebo, told nicotine and received placebo, told no nicotine and received nicotine, and told nicotine and received nicotine) as four indicator variables (told0-nic0, told1-nic0, told0-nic1, and told1-nic1) for each smoker group separately and by including a term in the regression of the form condition regressor for each indicator and regressor (Table S3 shows a complete list of regressors and fixed-effect coefficients). Specifically, if we let $l_{T0,t}$ be the indicator function for trials t where the subject has been told the cigarette does not contain nicotine, $l_{T1,t}$ be the indicator function for trials t where the subject has been told the cigarette contains nicotine, $l_{N0,t}$ be the indicator function for trials t where the subject receives a placebo cigarette, and $l_{N1,t}$ be the indicator function for trials t where the subject receives a cigarette with nicotine, then the model for subject j is given by

$$\begin{array}{l}{\tilde{b}}_{t+1,j}={\beta }_1\cdot 1_{T0,t}\cdot 1_{N0,t}+{\beta }_2\cdot 1_{T1,t}\cdot 1_{N0,t}+{\beta }_3\cdot 1_{T0.t}\cdot 1_{N1,t}+{\beta }_4\cdot 1_{T1,t}\cdot 1_{N1,t}\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left({\beta }_5\cdot 1_{T0,t}\cdot 1_{N0,t}+{\beta }_6\cdot 1_{T1,t}\cdot 1_{N0,t}+{\beta }_7\cdot 1_{T0,t}\cdot 1_{N1,t}+{\beta }_8\cdot 1_{T1,t}\cdot 1_{N1,t}\right)\cdot {\tilde{b}}_{t,j}\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left({\beta }_9\cdot 1_{T0,t}\cdot 1_{N0,t}+{\beta }_{10}\cdot 1_{T1,t}\cdot 1_{N0,t}+{\beta }_{11}\cdot 1_{T0,t}\cdot 1_{N1,t}+{\beta }_{12}\cdot 1_{T1,t}\cdot 1_{N1,t}\right)\cdot r_{t,j}\\ +{\left(Z_j\cdot u_j\right)}_t+{\epsilon }_t,\\ \end{array}$$

where the random effect for subject j is given by

$$\begin{array}{l}{\left(Z_j\cdot u_j\right)}_t=\left(u_{1,j}\cdot 1_{T0,t}\cdot 1_{N0,t}+u_{2,j}\cdot 1_{T1,t}\cdot 1_{N0,t}+u_{3,j}\cdot 1_{T0,t}\cdot 1_{N1},t+u_{4,j}\cdot 1_{T1,t}\cdot 1_{N1},t\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\left(u_{5,j}\cdot 1_{T0,t}\cdot 1_{N0,t}+u_{6,j}\cdot 1_{T1,t}\cdot 1_{N0,t}+u_{7,j}\cdot 1_{T0,t}\cdot 1_{N1,t}+u_{8,j}\cdot 1_{T1,t}\cdot 1_{N1,t}\right)\cdot {\tilde{b}}_t\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\left({\mathrm{u}}_{9,j}\cdot 1_{T0,t}\cdot 1_{N0,t}+u_{10,j}\cdot 1_{T1,t}\cdot 1_{N0,t}+u_{11,j}\cdot 1_{T0,t}\cdot 1_{N1,t}+u_{12,j}\cdot 1_{T1,t}\cdot 1_{N1,t}\right)\cdot r_t.\end{array}$$

Here $Z_j$ is the design matrix for the random effects, $u_j$ is the vector of random effects for subject j, $\widetilde{b_{t,j}}$ is the within-subject z-normalized (over the entire experiment for subject j) bet, ${\epsilon }_{t,j}\mathrm{\sim }N\left(0,{\sigma }^2\right)\hspace{0.5em}\mathrm{IID}$, $u_j\mathrm{\sim }N\left(0,\mathrm{\Sigma }\right),\hspace{0.5em}\mathrm{IID}$, and $\epsilon \hspace{0.5em}\mathrm{and}\hspace{0.5em}u$ are independent. Linear contrasts were then carried out to test the significance of differences between coefficients.

A similar regression was carried out separately for $T{D}_t$. The reward prediction error signal $T{D}_t$ is calculated as $\widetilde{g_t}-\widetilde{b_t}$, that is, the difference between the actual gain ($g_t=b_t{r}_t$) and the expected gain $\widetilde{b_t}$, where $\sim$ here means causal z score (z score over the bets/returns that have occurred up to and including time t), and the bet $b_t$ serves as the proxy for the expected gain. Table S5 includes a complete list of regressors and fixed-effect coefficients. The analyses were carried out in R (49) with the function lme (for the mixed regression) in the package lmer (50) and the function estimable (for the linear contrasts) in the package gmodels (51). Statistical significance was determined at P < 0.05, two tailed. The first bet and the last bet of each market were excluded to keep consistent with the fMRI analysis. Additionally, two smokers did not change their bets in one of the four sessions and the three corresponding sessions’ behavioral data were excluded from the behavioral analysis.

Hierarchical Bayesian model.

To further examine the possible effects of the autoregressive nature of the above multiple-regression analysis on the regression coefficients, we also analyzed the data using a full hierarchical Bayesian model in rjags (52). The model is specified hierarchically (code available upon request). At the top level we assume that for each subject i, in condition j (j = 1–4, corresponding to the 2 × 2 design) the normalized (see above) bet on trial k, $y_{i,j,k}$, given the normalized previous bet, the previous return, and parameters was distributed as

$$y_{i,j,k}|\mu \left(i,j,k\right),\tau \mathrm{\sim }\mathrm{Normal}\left(\mu \left(i,j,k\right),\tau \right),$$

where

$$\begin{array}{l}\mu \left(i,j,k\right)={\alpha }_0\left(i\right)+{\beta }_0\left(j\right)+{\alpha }_1\left(i\right)\cdot y_{i,j,k-1}+{\beta }_1\left(j\right)\cdot y_{i,j,k-1}+{\alpha }_2\left(i\right)\cdot r_{i,j,k-1}\\ +\hspace{0.17em}{\beta }_2\left(j\right)\cdot r_{i,j,k-1}.\end{array}$$

Here $\tau$ is the precision for the normal distribution. Next, at the coefficient level we assume for every subject i and condition j (and for each coefficient, indexes suppressed)

$$\begin{array}{l}\alpha \left(i\right)|\mathrm{tau}1\mathrm{\sim }\mathrm{Normal}\left(0,\mathrm{tau}1\right)\hfill \\ \beta \left(j\right)|b,\mathrm{tau}2\mathrm{\sim }\mathrm{Normal}\left(b,\mathrm{tau}2\right)\hfill \end{array}$$

and then finally for the last level

$$\begin{array}{c}\mathrm{tau}1\mathrm{\sim }\mathrm{gamma}\left(1,.1\right)\\ \mathrm{tau}2\mathrm{\sim }\mathrm{gamma}\left(1,.1\right)\\ b\mathrm{\sim }\mathrm{Normal}\left(0,.01\right).\end{array}$$

The sampler used three chains, 500 as the number of adaptive steps, 2,500 steps for burning in the chains, and 25,003 total saved steps. Note that for the two subjects for whom we had good data only in three conditions, we simply omitted the missing condition in the above.

Image Acquisition and Preprocessing.

The anatomical and functional imaging was conducted on a 3.0 Tesla Siemens Trio scanner at Baylor College of Medicine. High-resolution T1-weighted scans (1.0 × 1.0 × 1.0 mm) were acquired using an MP-RAGE sequence. Functional images were acquired using echo-planar imaging (EPI) and angled 30° with respect to the anterior–posterior commissural line. The detailed settings for the functional imaging were repetition time (TR) = 2,000 ms, echo time (TE) = 25 ms, flip angle = 90°, 37 slices, and voxel size = 3.4 × 3.4 × 4.0 mm. The average number of functional images acquired was 418 with a SD of 122. One fMRI run was acquired in each visit.

All imaging data were preprocessed using standard statistical parametric mapping (SPM8, Wellcome Department of Imaging Neuroscience) algorithms (fil.ion.ucl.ac.uk/spm). Functional images were motion corrected using a six-parameter rigid-body transformation to the first functional scan and unwarped using nonlinear basis functions. The mean functional images for each subject were coregistered to the subject’s high-resolution T1 structural scan, using a 12-parameter affine transformation. The subject’s T1 image was segmented into gray and white matter and then normalized using nonlinear basis functions to the Montreal Neurological Institute (MNI) template, and the functional images normalized to the template, with resampled 4 × 4 × 4-mm functional voxels. Functional images were smoothed spatially using an isotropic 8-mm full-width at half-maximum (FWHM) Gaussian kernel.

General Linear Modeling.

Event-related analyses of the fMRI data were conducted using SPM8. General linear modeling (GLM) (53) was conducted for the functional scans from each participant by modeling the observed BOLD signals and regressors to identify the relationship between the task events and the hemodynamic response. Regressors related to all visual and motor events were created by convolving a train of delta functions representing the sequence of individual events with the default basis function in SPM8, which consists of a synthetic hemodynamic response function composed of two gamma functions (53, 54).

We specified two separate GLMs for each subject. In both GLMs, we included the following regressors: (i) market type display, (ii) market history display, (iii) all key presses, (iv) market price reveal of round 1, (v) market price reveal of rounds 2–19, and (vi) market price reveal of round 20. Additionally, six parameters generated during motion correction were entered as covariates. In the first GLM, market value $r_t$ was entered as parametric regressors at the fifth regressor (market price reveal of rounds 2–19); in the second GLM, reward prediction error $T{D}_t$ was entered as parametric regressors at the fifth regressor (market price reveal of rounds 2–19). Regressors were orthogonalized in a standard SPM8 fashion. Linear contrasts of the parameter estimates were made to identify the effects of $r_t$ and $T{D}_t$ for each participant. These images from all participants were then entered into a second-level group analysis to implement a random-effects statistical model. A within-subject ANOVA model was conducted with the following factors: belief (told no nicotine vs. told nicotine) and cigarette (placebo vs. nicotine). Significant activations related to $r_t$ and $T{D}_t$ were identified at the level of P < 0.05 corrected for family-wise errors in conjunction with k > 2. Small volume correction (SVC) was used for regions of a priori interest, namely, bilateral striatum. SVC was conducted using a search volume of a sphere of 5 mm radius (640 mm³, 10 voxels) centered at [−12, 8, −6] and [12, 10, −6] for r and [−18, 0, −8] and [12, 8, −2] for TD, with masks based on striatum activations from an independent study as listed in Table S2 (39).

ROI Analysis.

We conducted ROI analysis, using an unbiased approach of defining striatum ROIs based on an independent fMRI study that included 63 healthy participants (age 32 ± 13 y, 34 females) who performed a similar sequential investment task (39). First, we identified the peaks of striatum activation related to the same regressor $r_t$ and $T{D}_t$ from this independent dataset: [−12, 8, −6] and [12, 10, −6] for $r_t$ and [−18, 0, −8] and [12, 8, −2] for $T{D}_t$ (Table S2). Second, using the MarsBaR toolbox (marsbar.sourceforge.net/), we created spherical ROIs with a 5-mm radius of bilateral ventral striatum centered at thee peaks. We also defined visual attentional ROIs related to market reveal as spheres with a 5-mm radius for the following regions: inferior temporal gyrus [48, −62, −2], inferior parietal lobule [30, −54, 48], and inferior frontal gyrus [52, 12, 20] (Table S7). Third, individual subject’s parameter estimates were then extracted from each ROI for each task condition and entered into paired t tests.