Impaired Bayesian Learning for Cognitive Control in Cocaine Dependence

Abstract

Background

Cocaine dependence is associated with cognitive control deficits. Here, we apply a Bayesian model of stop-signal task (SST) performance to further characterize these deficits in a theory-driven framework.

Methods

A “sequential effect” is commonly observed in SST: encounters with a stop trial tend to prolong reaction time (RT) on subsequent go trials. The Bayesian model accounts for this by assuming that each stop/go trial increases/decreases the subject’s belief about the likelihood of encountering a subsequent stop trial, P(stop), and that P(stop) strategically modulates RT accordingly. Parameters of the model were individually fit, and compared between cocaine-dependent (CD, n=51) and healthy control (HC, n=57) groups, matched in age and gender and both demonstrating a significant sequential effect (p<0.05). Model-free measures of sequential effect, post-error slowing (PES) and post-stop slowing (PSS), were also compared across groups.

Results

By comparing individually fit Bayesian model parameters, CD were found to utilize a smaller time window of past experiences to anticipate P(stop) (p<0.003), as well as showing less behavioral adjustment in response to P(stop) (p<0.015). PES (p=0.19) and PSS (p=0.14) did not show group differences and were less correlated with the Bayesian account of sequential effect in CD than in HC.

Conclusions

Cocaine dependence is associated with the utilization of less contextual information to anticipate future events and decreased behavioral adaptation in response to changes in such anticipation. These findings constitute a novel contribution by providing a computationally more refined and statistically more sensitive account of altered cognitive control in cocaine addiction.

1. INTRODUCTION

Cognitive control, the ability to withhold or modify actions in response to a dynamically changing environment, is a critical executive function. Using a variety of laboratory paradigms, numerous studies characterized deficits in cognitive control in chronic cocaine users (de Wit, 2009; Everitt et al., 2008; Garavan and Hester, 2007; Li and Sinha, 2008; Porrino et al., 2007). For instance, in the stop signal task, altered error processing predicted relapse and time to relapse in cocaine dependent individuals (Li et al., 2008b; Luo et al., 2013). A core process of cognitive control is to adjust behavior by learning from the changing contextual information. In the stop signal task, participants respond to a prepotent go signal, and withhold the response when they encounter an infrequent stop signal. Participants typically slow down in go trial reaction time (RT) following a stop error as compared to go trial – a phenomenon that has been called posterror slowing (Li et al., 2008b; Rabbitt, 1966). Acute administration of psychoactive substances such as amphetamine and alcohol diminishes post-error slowing (Bombeke et al., 2013; Wardle et al., 2012). Compared to healthy people, cocaine dependent individuals also demonstrated diminished post-error slowing (Li et al., 2006b), as in many other clinical populations that implicate deficits in cognitive control (Liu et al., 2013; Shiels et al., 2012).

In this behavioral study, we aimed to use a computational model to further characterize altered cognitive control in chronic cocaine users. We previously proposed a Bayes-optimal decision-making model for the stop-signal task (Shenoy et al., 2011; Shenoy and Yu, 2011), positing that participants choose to go or stop based on accumulating sensory evidence within a trial, as well as prior belief about the likelihood of a stop trial prior to stimulus onset. We showed (Shenoy et al., 2011; Shenoy and Yu, 2011) that this rational strategy explains classic stopping behavior, such as the increase in stop error rate with increasing stop-signal delay and faster stop error responses than correct go responses (Logan et al., 1984), as well as more subtle contextual effects such as the decrease in stop error rate and stop-signal reaction time when stop errors are penalized more (Leotti and Wager, 2009), and the decrease in stop error rate and increase in go RT when more stop trials are more expected (Emeric et al., 2007). In particular, by augmenting this decision-making model with trial-by-trial learning, we were able to account for the “sequential effect” in the stop-signal task (Ide et al., 2013; Shenoy et al., 2011): go RT slowing down after a run of stop trials and speeding up after a preponderance of go trials (Emeric et al., 2007; Li et al., 2008b).

We hypothesize that a core cognitive control deficit in cocaine addiction is impairment in learning from and adapting to changes in contextual information. That is, in the stop signal task, cocaine dependent participants may have an impaired ability to use contextual information to anticipate stop trials and/or to modulate within-trial processing as a function of prior anticipation; consequently, they would demonstrate a diminished sequential effect. Thus, our goal is to quantify and compare the sequential effect in a sample of cocaine dependent and healthy control individuals, and to identify the source of group difference by examining the Bayesian model parameters. We hope that, by addressing these issues, the current study will provide insights to the psychological processes that hinder behavioral adjustment in cocaine addicts.

2. MATERIAL AND METHODS

2.1 Subjects, Informed Consent, and Assessment

Fifty-one patients (37 men) with cocaine dependence (CD) and fifty-seven age and gender matched healthy control (HC) subjects (32 men) participated in this study (Table 1). CD participants were recruited from the local greater New Haven area via newspapers and flyers as part of a prospective study (Luo et al., 2013) and met criteria for current cocaine dependence, as diagnosed by the Structured Clinical Interview for DSM-IV (First et al., 1995). Of the 97 treatment-seeking CD participants recruited in the study of Luo et al. (2013), 51 demonstrated a significant sequential effect (please see Section 2.4 and Discussion) and were studied for the current work. Thus, the subjects represented a convenience sample as no power calculation was performed to predetermine the sample size. Recent cocaine use was confirmed by urine toxicology screens upon admission. They were drug-free while staying in an inpatient treatment unit at the Connecticut Mental Health Center during the study period. All subjects were physically healthy with no major medical illnesses or current use of prescription medications. None of them reported having a history of head injury or neurological illness. Other exclusion criteria included dependence on other psychoactive substances (except nicotine) and current or past history of psychotic disorders. Individuals with current depressive or anxiety symptoms requiring treatment or currently being treated for these symptoms were excluded as well. Treatment in the inpatient unit comprised individual counseling, cognitive behavioral and group therapy, as well as social service to facilitate their transition at discharge. The Human Investigation committee at Yale University School of Medicine approved the study, and all subjects signed an informed consent prior to participation.

Table 1.

Demographics and clinical characteristics of cocaine dependent (CD) and healthy control (HC) subjects

Subject characteristic	CD (n=51)	HC (n=57)	p-value
Age (years)	38.5 ± 7.9	37.3 ± 9.9	0.51*
Years of Education	11.8 ± 1.6	14.7 ± 2.8	3e-08*
Gender (M/F)	37/14	32/25	0.08^
Beck Depression Inventory (BDI)	13.9 ± 7.9	N/A	N/A
STAI State	40.1 ± 9.7	N/A	N/A
STAI Trait	41.9 ± 8.9	N/A	N/A
CCQ-Brief score	18.8 ± 7.2	N/A	N/A
Smoking Status (Yes/No)	34/17	20/37	0.001^
Years of alcohol use	15.2 ± 8.7	13.5 ± 15.3	0.50*
Days of drinking (prior month)	12.0 ± 8.7	3.6 ±5.3	1e-10*
Years of Marijuana use	10.0 ± 4.5	<1	1e-10*
Average amount of monthly cocaine use (gm) in the prior year	14.5 ± 20.6	N/A	N/A
Days of cocaine use in the prior month	15.1 ± 8.1	N/A	N/A
Years of cocaine use	15.7 ± 8.8	N/A	N/A
Days abstinent prior to assessment	18.2 ± 6.1	N/A	N/A

Note: values are mean ± S.D.; Symbols indicate the test applied in analysis:

^*two-tailed two-sample t test;

^{^}χ² test; Days of cocaine/alcohol use pertain to use prior to admission.

All CD participants were assessed with the Beck Depression Inventory (Beck et al., 1961) and the State-Trait Anxiety Inventory (Speilberger et al., 1970) at admission, both with scores within the range reported previously for individuals with cocaine dependence (Falck et al., 2002; Karlsgodt et al., 2003; Lopez and Becona, 2007; Rubin et al., 2007) (Table 1). Cocaine craving was assessed with the Cocaine Craving Questionnaire, brief version (CCQ-Brief), for all participants on the same day of the behavioral test (Sussner et al., 2006). The CCQ-Brief is a 10-item questionnaire, abbreviated from the CCQ-Now (Tiffany et al., 1993). It is highly correlated with the CCQ-Now and other cocaine craving measures (Sussner et al., 2006). Each item was rated on a scale from 1 to 7, with a higher total score (ranging from 10 to 70) indicating greater craving (Table 1).

Healthy control participants (HC) were drawn from the local community, underwent a thorough interview by a psychiatrist (C.-S. R. Li) to rule out a DSM-IV diagnosis including abuse of or dependence on a substance other than nicotine, and all tested negative for illicit substances in urine toxicology on the day of behavioral test. Smoking status and use of alcohol was documented. Previous use of marijuana for longer than one year or use of any other illicit substances were exclusion criteria. As none of the HC reported depression or anxiety symptoms that were deemed of clinical significance, an exclusion criterion, HC were not assessed with the BDI or STAI. None of the HC were under any psychotropic medications during the year prior to or at the time of behavioral test.

2.2 Behavioral task

We employed a simple reaction time task in this stop-signal paradigm (Hu et al., 2014; Li et al., 2006a; Li et al., 2005a; Logan et al., 1984). There are two trial types: “go” and “stop,” presented with an inter-trial interval of 2s, and occurring on each trial with 0.75 probability of being a go trial (0.25 probability stop trial). A small dot appears on the screen to engage attention at the beginning of a go trial. After a randomized time interval (fore-period) between 1 and 5 s, drawn from a uniform distribution, the dot turns into a circle (the “go” signal), prompting the subjects to quickly press a button. The circle vanishes at a button press or after 1 s has elapsed, whichever coming first, and the trial terminates. A premature button press prior to the appearance of the circle also terminates the trial. On a stop trial, an additional “X,” the “stop” signal, appears after and replaces the go signal, and instructs participants to withhold their response. Similar to go trials, a stop trial terminates at button press or 1 s after the appearance of the stop signal. Failure to withhold the go response for the 1 s constitutes a stop error. The stop signal delay (SSD) – the time interval between go and stop signals – starts at 200 ms and is adjusted according to a staircase procedure, increasing and decreasing by 67 ms each for a successful and failed stop (Levitt, 1971). Subjects were instructed to respond to the go signal quickly while keeping in mind that a stop signal could come up occasionally. The staircase procedure ensures that subjects would succeed in withholding their response in approximately half of the stop trials.

2.3 Analyses of behavioral performance in the stop signal task

We computed a critical SSD that represents the time delay between go and stop signals that a subject would need to succeed in 50% of the stop trials (Levitt, 1971). Specifically, SSDs across trials were grouped into runs, with each run defined as a monotonically increasing or decreasing series. We derived a mid-run estimate by taking the middle SSD (or average of the two middle SSDs when there was an even number of SSDs) of every second run. The critical SSD was computed by taking the mean of all mid-run SSDs. It was reported that, except for experiments with a small number of trials (less than 30), the mid-run estimate was close to the maximum likelihood estimate of X₅₀ (50% positive response; i.e., 50% SS in the SST, (Wetherill et al., 1966)). The stop signal reaction time (SSRT) was computed by subtracting the critical SSD from the median go trial RT (Logan, 1994).

It is known that in an RT task the RT of a correct response is prolonged following an error, compared with other correct responses, and this prolonged RT is thought to reflect error monitoring (Rabbitt, 1966). We thus computed the RT difference between the go trials that followed a stop error (SE) and those that followed another go trial, and termed the effect size of this RT difference “post-error slowing” (PES; Li et al., 2009a). Similarly, RT is prolonged following a high-conflict as compared to low-conflict trial. We thus computed the RT difference between the go trials that followed a stop trial and those that followed another go trial, and termed the effect size of this RT difference “post-stop slowing” (PSS; Li et al., 2009a).

2.4 Bayesian modeling of sequential effects

In the SST, the stop signal appears infrequently and elicits changes in the RT of go trials that follow a stop signal. Typically, compared to go trials that follow another go trial, the RT of a go trial following a stop trial is prolonged, reflecting a context-instructed behavioral adjustment. That is, participants slow down in order to accommodate the experience of a stop trial and the anticipation that a stop signal may appear again. While extant research including our own work has characterized this adjustment process as discrete events – that is, by comparing the RT of post-stop and post-go go trials – it is likely that participants use their experience with the extended sequence of events to update their belief in the likelihood of a stop signal continuously and adjust behavior accordingly. To capture how the series of events shape participants’ subjective estimate of the likelihood of a stop signal and their behavior, as indexed by RT, we need a mathematical model to describe the learning process.

As in our previous work (Ide et al., 2013), we use a dynamic Bayesian model (Yu and Cohen, 2009) to estimate the posterior belief of an impending stop signal on each trial, based on previous stimulus history and current observation. The Bayesian model assumes that subjects believe that stop signal frequency r_k on trial k has probability α of being the same as r_k-1, and probability (1-α) of being re-sampled from a fixed distribution π(r_k). Subjects are also assumed to believe that trial k has probability r_k of being a stop trial, and probability 1- r_k of being a go trial. Based on these generative assumptions, subjects are assumed to use Bayesian inference to update their prior belief of seeing a stop signal on trial k, p(r_k|s_k-1) based on the prior on the last trial p(r_k-1|s_k-1) and last trial’s true category (s_k=1 for stop trial, s_k=0 for go trial), where s_k={s₁,…, s_k} is short-hand for all trials 1 through k. Specifically, given that the posterior distribution was p(r_k-1| s_k-1) on trial k-1, the prior distribution of stop signal in trial k is given by:

$$p(r_k|{\mathit{s}}_{\mathit{k}\mathit{-}\mathit{1}})=\mathrm{\alpha }p(r_{k-\mathit{1}}|{\mathit{s}}_{\mathit{k}\mathit{-}\mathit{1}})+(1-\mathrm{\alpha })\mathrm{\pi }(r_k),$$

where the fixed distribution π(r_k) is assumed to be a beta distribution with two parameters, prior mean or pm, and scale or sc representing the shape parameter. The posterior distribution is computed from the prior distribution and the outcome likelihood according to the Bayes’ rule:

$$p(r_k|{\mathit{s}}_{\mathit{k}})\mathrm{\alpha }P(s_k|r_k)p(r_k|{\mathit{s}}_{\mathit{k}\mathit{-}\mathit{1}}).$$

The Bayesian estimate of the probability of trial k being stop trial, which we colloquially call P(Stop) in this paper, given the predictive distribution p(r_k |s_k-1) is expressed by:

$$P(s_k=1|{\mathit{s}}_{\mathit{k}\mathit{-}\mathbf{1}})=\int P(s_k=1|r_k)P(r_k|{\mathit{s}}_{\mathit{k}\mathit{-}\mathbf{1}}){\mathit{\text{dr}}}_k=\int r_{k}P(r_k|{\mathit{s}}_{\mathit{k}\mathit{-}\mathbf{1}}){\mathit{\text{dr}}}_k=(r_k|{\mathit{s}}_{\mathit{k}\mathit{-}\mathbf{1}}).$$

In other words, the probability P(Stop) of a trial k being a stop trial is simply the mean of the predictive distribution p(r_k | s_k-1). The assumption that the predictive distribution is a mixture of the previous posterior distributions and a generic prior distribution is essentially equivalent to using a causal, exponential, linear filter to estimate the current rate of stop trials (Yu and Cohen, 2009). In summary, for each subject, given a sequence of observed go/stop trials, and the three model parameters {α , pm , sc}, we estimated P(Stop) for each trial. Generally speaking, the α parameter quantifies the weight given by the subject to the previous trials, and pm is the mean of the fixed belief of stop signal. A sequential effect is defined as the linear correlation between P(Stop) and reaction time (RT) for all go trials.

That is, with assumption about participants’ prior belief in the likelihood of a stop signal, as modeled in a statistical distribution, and how participants learn to update this belief using a Bayes’ rule, we characterized the trial-by-trial change in their subjective state of stop signal anticipation and how this mental state impact their response time.

2.5 Sequential effect: a parameter set analysis

We hypothesized that CD and HC participants have different sequential effects. Therefore, we investigated, on an individual basis, the parameters of the Bayesian model that produced the maximum correlation, as indexed by coefficient R_max, between Go RT and P(Stop) (Ide et al., 2013). The search space of model parameters were set to the following ranges: α = [0.50, 0.51,…, 0.98], pm = [0.02, 0.03, …, 0.5], and sc = [1, 2,…, 12]. For each subject, we identified the best model parameter settings {α_max, pm_max , sc_max} that produced R_max. We compared CD and HC participants for α (the weight given to the dynamic, posterior distribution p(r_k-1|s_k-1), as opposed to the fixed prior distribution π), pm (mean of the beta distribution, which represents the individual’s fixed prior of stop trial occurrence), and sc (the scale parameter of the beta distribution, which reflects how skewed the distribution is around the mean).

2.6 Statistical comparison of CD and HC

Although individual model parameters were used in maximizing the sequential effect, we are aware of the uncertain and noisy nature of individual model estimates, considering the limited number of trials per subject. In this context, it is standard in model-based behavioral analyses (Camerer and Ho, 1999; Daw et al., 2006; O'Doherty et al., 2004) to use a fixed set of model parameters that are most representative and robust estimate of the group. Therefore, in comparing the sequential effect between CD and HC, we used the mean values of the best model parameters {α_max, pm_max , sc_max} as a fixed group parameter estimate, and computed the correlation between Go RT and P(Stop) for each individual participant, obtaining the “sequential effect” coefficient R_SeqEff. These coefficients R_SeqEff were Fisher transformed to Z_SeqEff to conform to a normal distribution and allow parametric analysis (Bond and Richardson, 2004). We also compared CD and HC in the slopes of linear regression between RT and P(Stop). Finally, the magnitude of sequential effect and model parameters were cross-correlated with clinical characteristics for CD. All statistical analyses were computed using MATLAB and Statistics Toolbox Release 2012b (The MathWorks, Inc., Natick, Massachusetts, United States).

3. RESULTS

3.1 Bayesian model parameters

In the first set of analyses, we examined the parameters of individual Bayesian models that produced the maximum correlation, as indexed by coefficient R_max, between Go RT and P(Stop). All participants demonstrated a significant R_max (p’s<0.01). The mean values of the best model parameters for CD and HD groups are summarized in Figure 1. The parameters pm_max (p>0.69) and sc_max (p>0.14) were not significantly different between the two groups. However, the parameter α_max was significantly greater in the HC (α_max=0.93±0.11) as compared to the CD group (α_max=0.84±0.19), corrected for Bonferroni multiple comparisons (T(106)=3.02, p=0.003<0.05/3). The results remained the same in an analysis of variance with age and gender as covariates (α_max: F(1)=9.23, p=0.003, pm_max: F(1)=0.21, p=0.6501, sc_max: F(1)=2.81, p=0.0967).

Figure 1.

Average values of the best model parameters: α_max (CD: 0.84±0.19, HC: 0.93±0.11), prior mean (CD: pm_max=0.19±0.22, HC: pm_max=0.17±0.21), and scale (CD: sc_max=9.4±4.4, HC: sc_max=10.5±3.3). Error bars indicate the standard deviation of the mean. *two sample t-test, p<0.05/3, Bonferroni corrected).

3.2 Sequential effect

Using the group averages of the Bayesian model parameters {α_max , pm_max , sc_max} = {0.93, 0.18, 10}, and {0.84, 0.18, 10} for the HC and CD groups, respectively, we computed the Pearson coefficient R_SeqEff, and normalized R_SeqEff to Z_SeqEff via Fisher transformation (see Section 2.6 for details). Z_SeqEff was significantly greater in the HC (Z_SeqEff =0.37 ± 0.12) as compared to CD (Z_SeqEff =0.31 ± 0.12) groups (two sample t-test, T(106)=2.38, p<0.019). Thus, CD participants showed greater RT variability that is unexplained by P(stop), in comparison to HC, who showed strong and consistent modulation of RT by P(stop). We performed another analysis to confirm this finding. We computed the Go RT for P(Stop) binned from 0.1 to 0.45 (equally spaced into 18 bins) for individual participants, and then averaged Go RT for each bin. We performed a linear regression each for CD and HC groups (R_SeqEff =0.81 with p<4.64e-05, and R_SeqEff =0.92 with p<5.5e-08, respectively) and the two regressions were significantly different in slope (T(32)=2.22, p<0.034; regression slope analysis (Zar, 1999); Figure 2).

Figure 2.

Sequential effect in cognitive control during the stop signal task. HC group (blue) presents a significantly steeper sequential effect, as measured by the correlation between go RT (ms, y-axis, left) and P(Stop), when compared to the CD (red) group (regression slope analysis, p<0.05). Error bars indicate standard error of the mean for each P(Stop) bin. The bottom histograms show the frequencies (numbers, y-axis, right) of go trials per bin of P(stop) each for HC (blue) and CD (red) groups. P(Stop) and RT data of these go trials are used to compute the sequential effect.

3.3 Correlation of sequential effect with stop signal task performance and clinical characteristics

We also computed traditional, non-model-based measures of behavioral performance in the stop signal task, as shown in Table 2. There were no significant differences on these outcome measures between CD and HC, except for a trend difference for %go, with correction for multiple comparisons (Table 2). The normalized sequential effect, measured by Z_SeqEff , was correlated to post-error slowing and post-stop signal slowing in HC but not CD participants (Figure 3). Z_SeqEff was not correlated to SSRT for HC or CD.

Table 2.

Performance in the stop signal task

	SSRT (ms)	Median go RT (ms)	%go	%stop	PES (effect size)	PSS (effect size)
CD (n = 51)	221 ± 48	568 ± 86	96.3 ± 1.6	51.5 ± 3.1	1.86 ± 1.54	2.17 ± 1.81
HC (n = 57)	209 ± 40	593 ± 96	97.1 ± 1.9	51.9 ± 2.4	2.28 ± 1.76	2.69 ± 1.76
p-value *	0.18	0.16	0.02	0.44	0.19	0.14

Note: All values are mean ± standard deviation; CD: individuals with cocaine dependence; HC: healthy controls; SSRT: stop signal reaction time; RT: reaction time; %go: percentage of go response trials; %stop: percentage of stop success trials; PES: post-error slowing; PSS: post-stop slowing.

^*p-value based on 2-tailed 2-sample t test.

Figure 3.

Stop signal task performance measures and sequential effect, as measured by Z_SeqEff. Left column: cocaine dependent (CD) group (n=51). Right column: healthy control (HC) group (n=57). SSRT: stop signal reaction time, PES: post error slowing, PSS: post-stop slowing. * PES and PSS are significantly correlated with Z_SeqEff in HC (p<0.05/6, Bonferroni corrected). The regression slopes of Z_SeqEff and PES were significantly different between HC and CD participants (T(104)=2.26, p=0.026, two-tailed Student t test). The regression slopes of Z_SeqEff and PSS were also significantly different between the two groups (T(104)=2.09, p=0.039, two-tailed Student t test).

Neither Z_SeqEff nor α_max was significantly correlated to years of education, years of alcohol use, or days of alcohol use in the prior month in CD or HC (all p’s > 0.05, corrected, pairwise correlation and multiple regression). In CD, neither Z_SeqEff nor α_max was significantly correlated to years of cocaine use, amount of monthly cocaine use in the prior year, days of cocaine use in the prior month, days of abstinence prior to the current assessment, BDI, STAI-S, STAI-T, or CCQ (all p’s > 0.05, corrected, pairwise correlation and multiple regression). In a two-by-two (group x smoker status) analysis of variance, we examined the influence of smoker status on the sequential effect. The results showed a significant main effect of group (F=4.46, p=0.037), as expected, but did not show a significant main effect of smoker status (p=0.666) or interaction effect (p=0.930).

4. DISCUSSION

4.1 Impaired Bayesian learning for cognitive control in cocaine addiction

Compared to demographically matched healthy participants, cocaine dependent individuals exhibited altered sequential learning for cognitive control, specifically using a diminished temporal window of contextual information to predict stopping events, as well as attenuated modulation of go RT by this contextual information. These new results reveal conceptually novel and mathematically precise deficits associated cocaine-addiction, enabled by the Bayesian modeling framework taken here. In other words, compared to control participants, cocaine dependent individuals did not use the contextual information as efficiently to update their estimate of the likelihood of a stop event and adjust their behavior accordingly. This novel finding on a deficit of learning has to our knowledge never been described in the context of cognitive control for stimulant dependence.

In the stop signal task, a traditional, non-model-based measure that resembles the sequential effect is post-error slowing (PES)/post-stop signal slowing (PSS), the extent of reaction time slowing in go trials following a stop error/stop trial, as compared to those following a go trial. We showed in earlier work (Ide et al., 2013), that the commonly observed post-error slowing is likely a consequence of post-stop slowing, as stop trials and error trials are highly correlated in the stop-signal task (see, however, Chang et al., 2014; Li et al., 2008b for neural correlates of post-error and post-success slowing). Once we take into account the increase in go RT as a function of increased P(stop), due to the observation of a stop trial, there is no additional modulation of go RT by whether that stop trial is an error or correct trial (Ide et al., 2013). Thus, PES is no more than a noisier form of PSS. In addition, PSS is a crude approximation (or a particular simple special case) of the Bayesian interpretation of sequential effect, the modulation of go RT by P(stop), as PSS takes into account only the last trial, while the Bayesian model takes into account all previous trials, with the fitted parameter α characterizing the effective time window over which past observations can affect the current trial. As such, PSS (and PES) has fundamentally less statistical power to uncover any group differences between CD and HC subjects. Indeed, we saw PSS/PES was not significantly different between the two groups, while the more sensitive Bayesian measures were. Looking closer, we saw that PSS was only significantly correlated with the Bayesian parameter Z_SeqEff in HC, and not in CD subjects. This suggests a potentially more random, or less principled, source of behavioral variability during cognitive control in cocaine-dependent users.

The current findings are consistent with reports from behavioral studies of probabilistic reversal learning, suggesting response perseveration and/or excessive switching in cocaine addicts (Ersche et al., 2011, 2008; Patzelt et al., 2014). In particular, Patzelt and colleagues used a Bayesian Hidden Markov model to distinguish between these two underlying processes in a normative framework (Patzelt et al., 2014). The latter study showed that deficits in reversal learning hinged primarily on increased responsivity to false feedback and spontaneous switching in cocaine addicts. Our results provide a potential explanation to this increased responsivity: compared to HC, CD participants utilize a smaller time window of past observations to predict the future, as measured by α in our Bayesian model. This tendency to discount past data and latch onto any potential novel pattern in the environment would naturally result in more spontaneous switching and sensitivity to unexpected feedback driven by noise rather than true changes in the environment. Separately, we speculate that the excessive responsivity of CD participants to (false) feedbacks may account for the seemingly intact PES/PSS along with a diminished sequential effect as observed with the stop signal task. Unlike a “one-shot” response, Bayesian sequential effect requires integration of trial experience in an internal model for behavioral guidance. This excessive responsivity to a salient stimulus and failure in evaluating and integrating its significance in the longer-term behavioral context is reminiscent of ill-adapted behavior such as impulsive, cue-elicited drug seeking in addicts (Broos et al., 2012; Moreno-Lopez et al., 2012; Perry et al., 2008). Also in support of this hypothesis are findings of altered activation of the medial prefrontal cortex in cocaine addicts (Li et al., 2008; Luo et al., 2013), including the dorsal anterior cingulate cortex (dACC) and supplementary motor area, regions that have been implicated in feedback based learning (Jocham et al., 2009). Indeed, lesioning of the dACC, while not impairing the performance of monkeys immediately after errors, made them unable to integrate risk and payoff in a dynamic foraging task (Kennerley et al., 2006). Furthermore, cocaine addiction is associated with altered error-related thalamic connectivity (Zhang et al., 2014), which plays a critical role in cognitive control (Hendrick et al., 2010; Ide and Li, 2011). It is possible that chronic cocaine use compromises catecholamine signaling required for contextual learning (Jocham and Ullsperger, 2009). On the other hand, as the magnitude of sequential effect did not correlate with any of the cocaine use variables, we cannot rule out the possibility that deficits in contextual learning may predispose individuals to cocaine misuse. Together, these results highlight the intricate nature of the deficits in cognitive control and the utility of examining these deficits under a normative, computational framework.

The Bayes model suggests that cocaine addicts learn from a shorter time window in anticipating future events than healthy humans. One possible mechanism underlying this difference is an impairment of working memory in cocaine addiction. Indeed, many previous studies have shown impaired working memory as a result of chronic exposure to cocaine (Albein-Urios et al., 2012; Fisk et al., 2011; Jovanovski et al., 2005; Kalapatapu et al., 2011; Kalechstein et al., 2013; Kubler et al., 2005; Lundqvist, 2005; Moeller et al., 2010; Porter et al., 2011; Rosselli and Ardila, 1996; Tau et al., 2013; Tomasi et al., 2007; Vonmoos et al., 2013). Since chronic cocaine use compromises cerebral structures critical for working memory and the extent of the structural deficits is related to years of drug use (Ide et al., 2014), more research is needed to confirm a lack of association between altered sequential effect and drug use. Furthermore, recent work suggests that a number of cognitive functions may recover during abstinence in cocaine addicted individuals (Connolly et al., 2013, 2012). Our cocaine participants were abstinent for only 2 to 3 weeks. It remains to be investigated whether they will recover from this impairment with a more extended period of abstinence.

The working hypothesis here, that dysfunctional cognitive control in cocaine addicts cooccur with an impaired ability to anticipate events that require modifications to ongoing behavior, is consonant with some existing modeling work on cognitive deficits in cocaine addiction. For example, it was recently proposed that cocaine addicts have enhanced model-free behavioral control and diminished model-based behavioral control, as compared to healthy controls (Lucantonio et al., 2014). Anticipatory or proactive control is a form of model-based control – thus, our work shows the impairment of a particular form of model-based control in the context of the stop-signal task. Although the statistical learning occurring in the stop-signal task, as captured by the Bayesian model in this work, is not driven by explicit rewards, and requires model-based mechanisms, cognitive deficits related to reward-driven learning need to be considered in future research (Redish, 2004).

4.2 Limitations of the study

There are a few important limitations to consider for the current work. First, CD and HC groups differed in a number of clinical characteristics that could not be accounted for with covariance analyses. Although these clinical variables, including years of education, did not appear to be correlated with the outcome measures in the CD group, we cannot rule out their influences on the main findings. Furthermore, our participants were not thoroughly assessed for impulsivity or ADHD symptoms; thus, the current results may not generalize to a wider population of cocaine addicts and warrant replication in future studies. Second, we did not observe a significant difference in SSRT between CD and HC in the current study, while previous work from our laboratory and others have largely reported a difference in SSRT between cocaine (and other substance) abusers and healthy controls. On the other hand, it should be noted that the effect size varied greatly across studies (please see (Smith et al., 2014) for a review). In our earlier work the effect size of the difference ranged from 0.098 to 0.641 (Bednarski et al., 2011; Li et al., 2008a, 2010, 2006b), suggesting a small to moderate difference. Additionally, the current study focused on individuals (including both HC and CD) who demonstrated a significant sequential effect. The rationale is that sequential effect represents the outcome of learning, which requires sufficient engagement and considerable attention devoted to the behavioral task. By focusing on those who demonstrated a significant sequential effect, we are able to identify what may have gone awry in the learning process in model-based analysis. The data of those who did not demonstrate a sequential effect will likely exhibit model fit with extreme parameters that are harder to interpret and to generalize across participants. Therefore, this group of CD individuals might not represent the typical population of cocaine addicts that we, and others, have examined. Third, cognitive control encompasses a wide range of psychological processes that go beyond the stop signal task. Studies that employ a different behavioral paradigm such as reversal learning and working memory may reveal deficits to complement the current findings. Finally, we did not observe a relationship between these deficits of proactive control and clinical characteristics including extent of cocaine use or treatment outcome measures. We also could not rule out the effects of alcohol (Bednarski et al., 2012; Hu et al., in press; Li et al., 2009b, 2005b; Yan and Li, 2009) or marijuana (Li et al., 2005b) use on the current results. Thus, the clinical significance of the current findings remains to be established.

4.3. Conclusions

In a moderate-sized sample tested on the stop signal task, cocaine addicts showed altered sequential effect, as compared to healthy control participants, suggesting a dysfunctional learning and adaptation system for cognitive control. Deficits in cognitive functioning hamper progress in treatment and recovery from addiction. The current study may provide a behavioral marker to monitor the effects of behavioral treatment and pharmacological regimens on enhancing cognitive functions of cocaine addicted individuals (McKee et al., 2007; Sofuoglu, 2010).

Highlights.

• We used a Bayesian model to examine cognitive control in cocaine dependence.

• Addicts showed less sequential effect and slower learning rate, compared to HC.

• Results suggest deficient utilization of contextual information in addicts.

• Findings provide a new account of altered cognitive control in cocaine addiction.