Addiction history associates with the propensity to form habits

Abstract

Learned habitual responses to environmental stimuli allow efficient interaction with the environment, freeing cognitive resources for more demanding tasks. However, when the outcome of such actions is no longer a desired goal, established stimulus-response (S-R) associations, or habits, must be overcome. Among people with substance use disorders (SUDs), difficulty in overcoming habitual responses to stimuli associated with their addiction in favor of new, goal-directed behaviors, contributes to relapse. Animal models of habit learning demonstrate that chronic self-administration of drugs of abuse promotes habitual responding beyond the domain of compulsive drug seeking. However, whether a similar propensity toward domain-general habitual responding occurs in humans with SUDs has remained unclear. To address this question, we used a visuomotor S-R learning and re-learning task, the Hidden Association Between Images Task (HABIT), which employs abstract visual stimuli and manual responses. This task allows us to measure new S-R association learning, well-learned S-R association execution, and includes a response contingency change manipulation to quantify the degree to which responding is habit-based, rather than goal-directed. We find that people with SUDs learn new S-R associations as well as healthy control subjects do. Moreover, people with an SUD history slightly outperform controls in S-R execution. In contrast, people with SUDs are specifically impaired in overcoming well-learned S-R associations; those with SUDs make a significantly greater proportion of perseverative errors during well-learned S-R replacement, indicating the more habitual nature of their responses. Thus, with equivalent training and practice, people with SUDs appear to show enhanced domain-general habit formation.

Introduction

Learned habitual responses to stimuli allow efficient navigation of daily life by allocating cognitive resources towards processes such as cognitive control, which enables flexible behavioral. However, when the outcome of such habitual actions is no longer a desirable goal, established stimulus-response (S-R) associations must be overcome. A definitive behavior of addiction is continued drug use despite serious negative consequences of such use. In essence, although the outcome of drug seeking and/or consumption is reduced in value from mostly positive to mixed or largely negative, these actions persist, and can be potently triggered by drug-associated cues. As such, addiction may be partially described as an initially goal-directed behavior that becomes a habit-based process as a consequence of reinforcement learning during repeated drug use (Balleine & O'Doherty, 2010; Belin, Belin-Rauscent, Murray, & Everitt, 2013; Belin, Jonkman, Dickinson, Robbins, & Everitt, 2009; Everitt & Robbins, 2005, 2013). Despite the clinical importance of understanding the maladaptively rigid behaviors that characterize substance use disorders (SUDs), investigation of behavioral rigidity in SUDs has been limited to date.

Data from animal models show that extended cocaine (Belin & Everitt, 2008; Zapata, Minney, & Shippenberg, 2010) or alcohol use (Corbit, Nie, & Janak, 2012; Dickinson, Wood, & Smith, 2002) promotes habitual behavior, suggesting that chronic exposure to drugs of abuse potentiates habitual responding more generally. In contrast to what is known in animals, relatively little is known about habit learning in humans, or whether addiction is associated with altered general capacity to learn or replace S-R associations, i.e. to form or break habits. Either enhanced habit formation or impaired ability to overcome habits could theoretically contribute to addiction.

Animal studies of S-R learning are typically limited to simple one-to-one mapping of stimuli onto response options, and while such designs have been used with humans (Deiber et al., 1997; Toni, Ramnani, Josephs, Ashburner, & Passingham, 2001) people learn such associations rapidly, limiting their use for examining learning over time and measuring transitions between goal-directed and habitual response selection. Moreover, habitual responding in animals is typically tested via outcome devaluation, with continued responding for a devalued outcome taken to indicate habitual responding (Dickinson, 1985). Outcome devaluation studies in humans replicate animal studies of simple S-R learning tasks (S. de Wit, Corlett, Aitken, Dickinson, & Fletcher, 2009; Tricomi, Balleine, & O'Doherty, 2009; Valentin, Dickinson, & O'Doherty, 2007), but such designs have substantial methodological limitations in humans. In particular, it is very difficult to identify multiple primary reinforcers equated for value across individuals that may then be devalued according to traditional animal paradigms. This difficulty precludes their use with special populations, which pose other recruitment challenges, and renders them ill-suited to multi-session tests of interventions to reduce habitual responding. Furthermore, these paradigms lack ecological validity for modeling stimulus-driven (i.e. habit-based) actions in the “real world.” During daily life, it is less often the case that the outcome of a no-longer adaptive action loses value; rather, it is that the outcome itself changes to one that is less (or un-) desirable. For example, the cue of walking into a darkened room will often trigger the automatic action of flipping the light switch. During a power outage, this action will yield no positive outcome, and yet this automatic action will persist despite its known lack of utility. In this case, the former outcome of this action (illumination of the darkened room) retains its value, but that is simply no longer the outcome of flipping the light switch. Likewise, in the case of maladaptive habit-based actions, such as compulsive drug use during a binge, the same action (e.g. lighting and smoking from a crack pipe) will no longer yield the initial outcome, a euphoric “high,” instead producing agitation and paranoid delusions. Again, the former outcome of this action (a euphoric high state) retains its value, but it is no longer the outcome of the action. Most work in humans to date has overcome these obstacles by instead employing probabilistic learning tasks (Dolan & Dayan, 2013). However, while such paradigms are useful in investigating the ability to flexibly adapt to dynamic response contingencies, these paradigms cannot promote enduring habitual responses to stimuli. Our task, while simplistic, provides a useful laboratory-based model of these sort of stimulus-response-outcome contingency changes that are a natural part of human life.

Few studies to date have investigated the relationship between habitual behavior and drugs of abuse in humans. First, adult smokers demonstrate both goal-directed and habitual responding for natural rewards and cigarettes, dependent upon age, smoking habit severity, and impulsiveness (Hogarth & Chase, 2011; Hogarth, Chase, & Baess, 2012). Second, a recent neuroimaging study in alcohol-dependent patients, including those concurrently using psychoactive medications for depression and anxiety disorders, found preferential habit-based responding during task performance at the expense of goal-directed behavior (Sjoerds et al., 2013); however, confounding factors preclude strong linkage between habitual responding and alcohol use disorders.

Finally, no published work to date has investigated the transition between goal-directed and habitual response selection during S-R formation or the replacement of S-R associations in people with SUDs. To address this knowledge gap, we compared S-R association learning and replacement between healthy adults with no SUD history, and currently abstinent people with a lifetime SUD diagnosis (Table 1). We predicted that an SUD history would associate with enhanced capacity to acquire new S-R associations and impaired ability to replace established responses, with a specific increase in perseverative responding when attempting to change established S-R associations. To test these ideas, we employed the Hidden Association Between Images Task (HABIT, Figure 1), a visuomotor S-R learning and re-learning task with abstract visual stimuli and manual responses. As the behavioral data were not normally distributed, we applied generalized linear mixed-effects models (GLMMs) to characterize the change in behavioral performance over time, evaluating whether SUD status uniquely accounts for significant variability in learning and/or re-learning trajectories across individuals.

Table 1. Sample Demographics and Psychometric Data.

	CS (n=40)	SUD (n=22)	t(60)	p value
Demographics
Age (yrs)	24 ± 6	29 ± 6	-2.65	0.01
SILS (calculated) IQ	105 ± 6	99 ± 6	4.32	<0.001
Education (yrs)	15 ± 2	15 ± 2	0.40	0.69
SES	46 ± 9	41 ± 12	1.51	0.14
Gender (% female)	50	16		ns†
Ethnicity (% non-white)	27	43		0.28#
Substance Use related
AUDIT Total	4 ± 3	23 ± 10	-8.60	<0.001§
AUDIT Consumption	3 ± 2	8 ± 3	-7.28	<0.001
AUDIT Dependence	0.08 ± 0.35	6 ± 5	-8.31	<0.001§
AUDIT Harm	0.78 ± 1.33	8 ± 6	-7.19	<0.001§
DAST	1 ± 1	17 ± 7	-10.79	<0.001§
DUSI-I (%)	0.10 ± 0.12	0.80 ± 0.18	-16.25	<0.001§
FTQ density (%)	0.16 ± 0.22	0.41 ± 0.23	-4.28	<0.001
Psychometric
BDI	3 ± 4	6 ± 6	-1.69	0.10§
BIS Total	55 ± 8	68 ± 15	-3.61	0.001§
BIS Attention	14 ± 3	18 ± 5	-2.55	0.01
BIS Motor	21 ± 3	25 ± 6	-2.73	0.01§
BIS Non-planning	20 ± 4	25 ± 6	-4.31	<0.001§
LOC	10 ± 3	8 ± 4	1.99	0.051
MMPI-Antisocial Practices Scale	6 ± 3	10 ± 5	-3.57	0.001§
STAI Total	60 ± 15	67 ± 15	-1.63	0.11
STAI-State Anxiety	27 ± 7	29 ± 7	-0.96	0.34
STAI-Trait Anxiety	33 ± 8	37 ± 9	-2.02	0.048
TAF Total	17 ± 13	19 ± 14	-0.43	0.67
TAF Moral	16 ± 11	15 ± 10	0.40	0.69
TAF Self	1.2 ± 2.1	3.0 ± 3.6	-2.15	0.03§
TAF Others	0.6 ± 1.6	1.4 ± 3.1	-1.14	0.26§

Values are reported as mean ± standard deviation. Reported p-values reflect the results of unpaired two-tailed comparison between groups. SUD, History of substance use disorder subject; CS, Control subject; IQ, Intelligence Quotient; SES, Socioeconomic Status; AUDIT, Alcohol Use Disorders Identification Test; DAST, Drug Abuse Screening Test; DUSI-I, Drug Use Screening Inventory, Domain I; FTQ, Family Tree Questionnaire; BDI, Beck Depression Index; BIS, Barratt Impulsivity Scale; LOC, Locus of Control; MMPI, Minnesota Multiphasic Personality Inventory; SILS, Shipley Institute of Living Scale; STAI, State-Trait Anxiety Inventory; TAF, Thought Action Fusion Scale. Boldface indicates significant values.

^§p-value represents results from Satterthwaite method for unequal variances.

^†p-value represents results of χ² test. ns: p>0.05.

^#p-value represents result of Fischer's exact test.

Figure 1. Diagram of HABIT paradigm structure.

(A) Panel depicts training session (“Session 0”) and Test session (“Session 1”), which occurs on a subsequent day. Session 1 is divided into Part 1 (pre-response change; 6 runs) and Part 2 (post-response change; 6 runs). (B) Task schematic for Part 1 (left) and Part 2 (right) of the HABIT Test Session.

Methods

Participants

A total of 62 subjects were recruited from the University of North Carolina at Chapel Hill (UNC) and the surrounding community via advertisements. Subjects were recruited into two groups, based on whether they did (n=22 SUD) or did not (n=40 control; Ctrl) meet DSM-IV criteria for past drug or alcohol dependence in a structured clinical interview (n=7 alcohol, n=4 opiates, n=11 stimulants, of which n=13 were poly-substance abusers) (Sheehan et al., 1998). SUD participants self-reported a minimum of 2 weeks of abstinence at the time of recruitment (M = 2 ± 2.5 yrs). All subjects were healthy individuals 18-40 years old with no known history of neurological disorders, no current psychiatric diagnoses (n=5 SUDs met criteria for past depression) or psychoactive drug or medication use (excluding nicotine and caffeine), and reported an IQ within the normal range (≥80). Participants were screened for psychoactive drug use (Biotechnostix, Inc., Markham, ON), including alcohol (FC-10, Lifeloc Inc., Wheat Ridge, CO) in each session. Thirteen additional participants were recruited, but failed to complete the training session (see “Behavioral Task”), and another 12 participants failed to return for or complete the testing session. As expected, the SUD and control groups differed significantly in terms of substance use, with higher scores on all measures, including family history of alcohol abuse in the SUD group (Table 1). The SUD and control groups did not differ significantly in terms of education, socioeconomic status, gender or ethnicity, but did differ in terms of age and estimated IQ, with significantly lower average IQ and higher average age for the SUD group relative to the Ctrl group (Table 1); to control for these differences, age and IQ were included as covariates in all analyses. Each subject provided written informed consent as approved by the UNC Office of Human Research Ethics.

General Procedure

Subjects participated in 2 sessions, with at least 1 night's sleep between the first and second sessions. Subjects were paid for their participation, including performance bonuses in the second (testing) session. During session 0, participants first underwent a structured clinical interview, then completed a battery of standard questionnaires (see “Behavioral Inventories”), followed by behavioral training on the computerized S-R learning task (see “Behavioral Task”). Learning and habitual responding was then tested during Session 1.

Behavioral Inventories

We administered a number of standard questionnaires to quantify factors that could impact our results. We quantified alcohol use behavior with the Alcohol Use and Disorders Identification test (AUDIT) (Saunders, Aasland, Babor, de la Fuente, & Grant, 1993) and substance use behavior with the Drug Use Screening Inventory, Domain I (DUSI-I) (Tarter, 1990) and the Drug Abuse Screening Test (DAST) (Skinner, 1982). We calculated density of familial alcohol abuse using the Family Tree Questionnaire (FTQ) (Mann, Sobell, Sobell, & Pavan, 1985). Neuropsychological questionnaires included the Barratt Impulsivity Scale (BIS-11) (Barratt, 1994), the Beck Depression Inventory (BDI) (Beck & Steer, 1987), Rotter's Locus of Control scale (LOC) (Rotter, 1966), the State-Trait Anxiety Inventory (STAI) (Spielberger, 1985), the Thought Action Fusion scale (TAF) (Shafran, Thordarson, & Rachman, 1996) and the Antisocial Practices (APS) of the Minnesota Multiphasic Personality Inventory 2 (MMPI-2) (Butcher, Graham, Williams, & Ben-Porath, 1990). Education and occupation were quantified with the Hollingshead Socioeconomic status (SES) score (Hollingshead, 1975). We estimated IQ with the Shipley Institute of Living Scale (SILS) (Zachary, 1991).

Behavioral Task

The Hidden Association Between Images Task (HABIT) is a stimulus-response (S-R) learning and re-learning task implemented in E-Prime 2.0 (PST Inc., Pittsburgh, PA) comprised of a HABIT Training Session and a two part HABIT Test Session, which occurs on a subsequent day (Fig. 1). The training and test session Part 1 have been previously described in detail (Boettiger & D'Esposito, 2005). In brief, stimuli were presented on a color LCD screen, and subjects used a four-button keypad for manual response selection using the fingers of their dominant hand. Participants were given instructions and a brief familiarization prior to completing the training phase of the task. Participants viewed abstract visual stimuli displayed briefly (700 ms) on the screen that they learned, through trial and error, to associate with specific manual responses. During the first, training session, participants learned two sets of S-R rules (Familiar) to a criterion of ≥ 90% accuracy. Participants then returned after ≥1 night's sleep to complete the test session. In the second, testing session, participants first demonstrated retention of the previously learned (Familiar) associations, then the learning task (HABIT Test Part 1; Fig. 1) began. In the learning task, blocks of the two Familiar sets were interspersed with blocks composed of two new (Novel) stimulus sets, to measure new S-R learning, and blocks of a control condition, consisting of novel, unrelated stimuli (No Rule); blocks consisted of 15 randomly selected stimuli from the relevant set. Following 6 “runs” of 15 blocks each (3 per set type), subjects were informed that the correct responses for two sets (one Familiar and one Novel set) had changed (HABIT Test Part 2; Fig. 1). As the previously correct responses for the changed sets produce a negative rather than positive outcome, one could construe this change in response contingency as a response “devaluation,” although this manipulation is quite different from the outcome devaluation procedures traditionally used in studies of habitual responding. Devaluing outcomes is methodologically tricky in human studies, as primary rewards are not universally palatable. Moreover, points (or other performance metrics), or money tend to remain intrinsically rewarding and are difficult to realistically devalue. Participants then learned the new correct S-R associations through trial and error. This “response devaluation” manipulation allows us to quantify habitual responding when attempting to overcome both well-learned (Familiar) and freshly learned (Novel) S-R associations, as the proportion of perseverative errors can be taken as an index of the degree to which responses are outcome independent (i.e. habit-based), as opposed to outcome-driven (i.e. goal-directed). By introducing S-R changes for both Familiar and Novel sets, at a point where performance is approximately equivalent, we can rule out performance deficits due to impaired response inhibition. Moreover, including Familiar and Novel sets in which correct responses do not change allows us to control for effects on performance of time and of context change.

Data Analysis

Our main index of performance was number of correct responses out of total responses across both epochs of the task (6 runs each, pre- and post-contingency change). Our data structure is composed of 48 repeated measures, consisting of 4 stimulus set types (2 Familiar, 2 Novel) that are measured within person over the 12 time points. We also collected reaction time data in each trial, and were able to categorize error types (perseverative button press, other incorrect button press) post-contingency change to distinguish between habit-based and goal-directed response strategies. Due to the non-normal nature of these data, rather than using a mixed model repeated-measures ANOVA analytical approach, we instead used a Generalized Linear Mixed-Effects Model (GLMM) with a binomial distribution and logit link function, which models a linear rate of learning and is ideally suited to account for the non-linear nature of learning rates in terms of probability. Our GLMM approach is described in detail in the next section. To test the significance of between group comparisons for demographic and psychological variables, we used unpaired two-tailed t-tests for continuous measures and χ² tests for categorical measures. Additionally, we used a one-way ANOVA to test for statistically significant differences between groups in perseverative responding, and the nonparametric Kruskal-Wallis test to compare perseverative responding among SUD subgroups. All analyses included age and IQ as covariates. All data analyses were performed within SAS (Cary, NC).

Specification of Generalized Linear Mixed-Effects Models

Performance data in this S-R learning task were the number of correct responses within each block, a non-normally distributed outcome heavily skewed towards the top end of possible values, hence performance accuracy was characterized by fitting GLMMs with a binomial distribution and logit link function. GLMMs provide a statistically efficient way to independently account for variance at different levels within nested data. In the present instance, repeated measures (performance within 6 runs each during the learning and re-learning epochs) are nested within persons, and thus GLMMs could be specified to account for both within- and between-person variability in performance accuracy. Our data structure is composed of 48 repeated measures, consisting of 4 stimulus set types (2 Familiar, 2 Novel) that are measured within person over the 12 time points. The measurement of accuracy at multiple time points both pre- and post-contingency change yields increased power to detect between-subject differences in within-subject change, with particular emphasis on the ability to compare pre- and post-contingency change trajectories and to capture changes in performance over time. For each set type, we modeled the timecourse of performance over each epoch (pre- and post-contingency change) independently and also included additional unique variables capturing the change in performance following the S-R contingency change manipulation. Our analytic approach involved first fitting a baseline model (Model 1) to represent changes in performance accuracy during the learning and re-learning epochs as a function of set type and changed response contingencies, controlling for age and IQ. Next, we added SUD status as a predictor of performance (Model 2), and conducted a likelihood ratio test to evaluate improvement in model fit. Models were estimated using maximum likelihood in the GLIMMIX procedure of SAS 9.3, implemented using adaptive quadrature with nine quadrature points per dimension of integration.

Defining π_ij to be the probability that person j will produce an accurate response to a stimulus given during run i, Model 1 was specified as:

$$\text{logit}({\pi }_{ij})={\beta }_{0ij}+{\beta }_{1ij}\mathit{\text{Trend}}{1}_{ij}+{\beta }_{2ij}{\mathit{\text{Dropoff}}}_{ij}+{\beta }_{3ij}{\mathit{\text{ChangeTrend}}}_{ij}+{\beta }_4{\mathit{\text{Age}}}_j+{\beta }_5{\mathit{\text{IQ}}}_j+u_{0j}+u_{1j}\mathit{\text{Trend}}{1}_{ij}+u_{2j}{\mathit{\text{Dropoff}}}_{ij}+u_{3j}{\mathit{\text{ChangeTrend}}}_{ij}$$ Eqn. 1

where fixed effects are designated by β and the first four fixed effects, which capture changes in performance accuracy over time and across conditions, are decomposed as follows

$$\begin{array}{c}{\beta }_{0ij}={\beta }_{00}+{\beta }_{01}{\mathit{\text{Set}}}_{ij}\\ {\beta }_{1ij}={\beta }_{10}+{\beta }_{11}{\mathit{\text{Set}}}_{ij}\\ {\beta }_{2ij}={\beta }_{20}+{\beta }_{21}{\mathit{\text{Set}}}_{ij}+{\beta }_{22}{\mathit{\text{NewResponse}}}_{ij}+{\beta }_{23}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{NewResponse}}}_{ij}\\ {\beta }_{3ij}={\beta }_{30}+{\beta }_{31}{\mathit{\text{Set}}}_{ij}+{\beta }_{32}{\mathit{\text{NewResponse}}}_{ij}+{\beta }_{33}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{NewResponse}}}_{ij}\end{array}$$ Eqn. 2

Last, the random effects are designated by u and assumed to be normally distributed with a full covariance matrix G.

The variables within the model were coded to enhance interpretation of the parameter estimates. The covariates Age and IQ were mean-centered so that all fixed effects could be interpreted to represent effects for a participant of typical Age and IQ. Trend1 was coded −5, −4, …, 6 for the twelve runs; Dropoff was coded zero for runs occurring during the learning epoch and one for runs during the re-learning epoch; and ChangeTrend was coded zero for runs occurring during the learning epoch and 1, 2, …, 6 for runs during the re-learning epoch. Given this coding, β_0ij represents performance at the final run of the learning epoch; β_1ij represents the increase in accuracy over the learning epoch; β_2ij represents the drop off in accuracy between the last run of the learning epoch and the first run of the re-learning epoch due to changes in S-R contingencies; and β_3ij indicates the difference in rate of improvement in accuracy in the re-learning epoch relative to the learning epoch. Eqn. 2 shows that the values of these four coefficients were a function of set type (Set; coded one for Familiar, zero for Novel) and whether the response was devalued in the re-learning epoch (NewResponse; coded one for devalued sets and zero for non-devalued sets). Additionally, the random effects in Eqn. 1 allowed for person-to-person variability in the four components of the performance accuracy trajectories.

Model 2 retains Eqn. 1 but includes SUD status as a predictor such that Eqn. 2 is elaborated as follows:

$$\begin{array}{c}{\beta }_{0ij}={\beta }_{00}+{\beta }_{01}{\mathit{\text{Set}}}_{ij}+{\beta }_{02}{\mathit{\text{SUD}}}_j+{\beta }_{03}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{SUD}}}_j\\ {\beta }_{1ij}={\beta }_{10}+{\beta }_{11}{\mathit{\text{Set}}}_{ij}+{\beta }_{12}{\mathit{\text{SUD}}}_j+{\beta }_{13}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{SUD}}}_j\\ {\beta }_{2ij}={\beta }_{20}+{\beta }_{21}{\mathit{\text{Set}}}_{ij}+{\beta }_{22}{\mathit{\text{NewResponse}}}_{ij}+{\beta }_{23}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{NewResponse}}}_{ij}+{\beta }_{24}{\mathit{\text{SUD}}}_j+{\beta }_{25}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{SUD}}}_j+{\beta }_{26}{\mathit{\text{NewResponse}}}_{ij}\times {\mathit{\text{SUD}}}_j+{\beta }_{27}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{NewResponse}}}_{ij}\times {\mathit{\text{SUD}}}_j\\ {\beta }_{3ij}={\beta }_{30}+{\beta }_{31}{\mathit{\text{Set}}}_{ij}+{\beta }_{32}{\mathit{\text{NewResponse}}}_{ij}+{\beta }_{33}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{NewResponse}}}_{ij}+{\beta }_{34}{\mathit{\text{SUD}}}_j+{\beta }_{35}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{SUD}}}_j+{\beta }_{36}{\mathit{\text{NewResponse}}}_{ij}\times {\mathit{\text{SUD}}}_j+{\beta }_{37}{\mathit{\text{Set}}}_{ij}\times {\mathit{\text{NewResponse}}}_{ij}\times {\mathit{\text{SUD}}}_j\end{array}$$ Eqn. 3

Models 1 and 2 are nested in their fixed effects, permitting a likelihood ratio test of the overall effect of SUD status on performance accuracy trajectories (Table 2).

Table 2. Fixed Effect Estimates (Top) and Variance-Covariance Estimates (Bottom) for Models of the Predictors of Learning Behavior.

Parameter	Model 1	Model 2
	Fixed effects
Intercept	0.95**(0.08)	0.91**(0.10)
Set	0.33**(0.03)	0.24**(0.04)
Trend1	0.23**(0.02)	0.23**(0.02)
Set×Trend1	-0.12**(0.01)	-0.12**(0.01)
Dropoff	-0.53**(0.07)	-0.48**(0.08)
Set×Dropoff	-0.01(0.06)	0.02(0.07)
Dropoff×NewResponse	-0.66**(0.04)	-0.69**(0.05)
Set×Dropoff×NewResponse	0.33**(0.06)	0.39**(0.07)
ChangeTrend	-0.13**(0.02)	-0.15**(0.02)
Set×ChangeTrend	0.07**(0.02)	0.10**(0.02)
NewResponse×ChangeTrend	0.11**(0.01)	0.15**(0.02)
Set×NewResponse×ChangeTrend	-0.01(0.02)	-0.03(0.03)
Age (centered)	0.01(0.01)	0.01(0.01)
IQ (centered)	0.002(0.01)	0.01(0.01)
SUD		0.13(0.18)
Set×SUD		0.28**(0.07)
Trend1×SUD		0.02(0.03)
Set×Trend1×SUD		0.01(0.02)
Dropoff×SUD		-0.14(0.13)
Set×Dropoff×SUD		-0.11(0.13)
Dropoff×NewResponse×SUD		0.10(0.09)
Set×Dropoff×NewResponse×SUD		-0.17(0.13)
ChangeTrend×SUD		0.04(0.04)
Set×ChangeTrend×SUD		-0.09*(0.04)
NewResponse×ChangeTrend×SUD		-0.09*(0.03)
Set×NewResponse×ChangeTrend×SUD		0.08(0.04)
	Variance of Random Effects
Intercept	0.40	0.38
Trend1	0.01	0.01
Changetrend	0.01	0.01
Dropoff	0.16	0.15
	Correlations Between Random Effects
Trend1/intercept	0.05	0.05
Changetrend/intercept	-0.04	-0.04
Changetrend/trend1	-0.01	-0.01
Dropoff/intercept	-0.14	-0.13
Dropoff/trend1	-0.03	-0.02
Dropoff/changetrend	0.01	0.01
-2* log-likelihood	23,279.02	23,207.43**

Standard errors are in parentheses. Set denotes the familiar versus novel set-type variable, with novel set-type as the reference category. Trend1 indicates the slope of performance during pre-contingency change time points. Drop-off signifies the difference in performance pre- and post-contingency change. NewResponse indicates a change in the correct response as a result of devaluation. Changetrend is the variable denoting the change in post-change performance relative to pre-change performance. SUD, substance use disorder. The random parameters represent the variance and covariance estimates generated from inclusion of random effects in the model. The -2log-likelihood demonstrates the value for model fit.

^*p<0.05,

^**p<0.001.

Results

Participants learned two sets of (Familiar) S-R associations during an initial HABIT training session, then returned for a HABIT testing session broken into two epochs: an initial learning epoch in which participants both executed the previously learned (Familiar) S-R associations and learned two new (Novel) sets of S-R associations, and a subsequent “re-learning” epoch, in which the established S-R contingencies for one of the Familiar S-R sets and one of the Novel S-R sets changed (Fig. 1). During the re-learning epoch, the previously correct response to the stimuli in the changed sets is met with a punishment instead of a reward, reducing the value of selecting the previously learned action in response to those stimuli; this change in response contingency allowed us to quantify perseverative errors as an index of habitual responding. The learning and re-learning epochs were each divided into six segments, each in turn comprised of 3 randomly ordered blocks of S-R set types (18 trials per block). Thus, at the onset of the re-learning epoch, participants have completed 324 trials for each of the Novel S-R sets, and approximately 3 times as many trials for the Familiar sets [average: 1038 trials (95% C.I.: 952, 1124)].

Behavioral performance during training session

During the initial training session, subjects were required to reach a performance criterion of 90% accuracy for each S-R set. These sets are designated as Familiar in the subsequent learning and re-learning epochs. Set order was counterbalanced across participants and set order did not differ between groups, χ²₍₁₎ = 0.40, p=0.53. Training to criterion took an average of ∼25 min, with no significant difference between groups in the number of blocks to criterion (Ctrl: 11 blocks; SUD: 9 blocks; 40 trials per block; F_(3,56)=0.67 p=0.57). Learning the associative rules for the second S-R set was always more rapid, and also did not differ significantly between groups (Ctrl: 4 blocks; SUD: 4 blocks; F_(3,56)=0.39 p=0.76). Thus, prior to returning for the testing session, training performance between groups was equivalent. Moreover, the time between the training and testing sessions did not differ significantly between groups (Ctrl: 10 days; SUD: 8 days; t₍₆₀₎= 1.09, p=0.28).

Behavioral performance during testing session — Model 1: Baseline Model without SUD Status

We found no significant main effects of age or IQ in the baseline GLMM fit to the performance data (Table 2). During the learning epoch, we observed a significant interaction between set-type and time prior to S-R contingency changes (p<0.001; “Set×Trend1”, Table 2), indicating that, as expected, performance improved more over time in the Novel S-R sets relative to the Familiar S-R sets. Somewhat surprisingly, the performance drop-off effect after contingency change was greater for Novel S-R sets with changed responses contingencies relative to Familiar S-R sets with changed responses contingencies (p<0.001; “Set×Dropoff×NewResponse”, Table 2).

During the re-learning epochs the difference in learning rate interacted with S-R set-type, with unchanged Novel S-R sets showing a shallower rate of improvement (p<0.001; “ChangeTrend”, Table 2), which was less pronounced for unchanged Familiar S-R sets (p<0.001; “Set×ChangeTrend”, Table 2). This difference in learning rate between epochs also differed between S-R sets with changed versus unchanged response contingencies (p<0.001; “NewResponse×ChangeTrend”, Table 2). For changed S-R sets, the rate of re-learning was steeper than that observed during the learning epoch, in contrast to the shallower re-learning rate for unchanged sets.

Model including SUD Status

Across both epochs, a model including group as a performance predictor (Model 2, Table 2) fit the data significantly better than did an identical model excluding group as a predictor (Model 1, Table 2; p<0.001). This result indicates that presence or absence of an SUD history accounted for significant variability in HABIT performance across individuals. As described below, this result does not reflect a performance deficit in the SUD group.

To further dissect HABIT performance, we first evaluated the initial learning epoch. Task performance improved over the course of the epoch, with greater improvement for the Novel S-R sets (Fig. 2; p<0.001, “set×trend1”, Table 2). As shown in Figure 2, participants executed Familiar S-R sets more accurately than Novel S-R sets, a distinction that was heightened in the SUD group. The groups did not differ in terms of performance improvement during the initial learning epoch (p=0.50; “set×trend1×SUD”, Table 2), but an SUD history predicted more accurate execution of Familiar S-R sets (Fig. 2, magenta lines; p<0.001; “set×SUD”, Table 2). Thus, an SUD history predicts intact ability to form new S-R associations, and a somewhat heightened ability to accurately execute established S-R associations.

Figure 2. Mean accuracy values during pre-change task performance.

Solid lines represent the control group (Ctrl) and dashed lines represent the SUD history group. Mixed models demonstrated that group status significantly predicts accuracy (Table 2). Familiar (magenta) performance starts high and remains high as performance progresses, with SUD history predicting more accurate execution of S-R associations (Model 2, Table 2). Novel (yellow) set performance improves over time as S-R associations are learned, with no difference between groups in rate of learning (Table 2). (A) Data plots depict raw accuracy values adjusted for Age and IQ; error bars represent within subject standard error of the mean. (B) Corresponding model predicted values.

At the outset of the re-learning epoch, performance immediately declined for all sets in both groups as shown in Figure 3 (right panels). As in Model 1, the changed-unchanged S-R contingency contrast was more pronounced in the Novel S-R sets (yellow) relative to the Familiar S-R sets (magenta; p<0.001; “Set×Dropoff×NewResponse”, Table 2); SUD status did not significantly interact with these parameters (Table 2). This finding indicates that both groups show evidence of overtraining in the Familiar S-R sets relative to the Novel S-R sets, which is reported to facilitate reversal learning for S-R tasks (McLaren et al., 2014). This is consistent with the fact that participants completed 2-3 times as many trials for the Familiar S-R sets relative to the Novel S-R sets (n=324 trials per set). As is evident in Figure 3, performance improved over the course of the re-learning epoch, with shallower rates of increase relative to the initial learning epoch for unchanged, Novel sets (p<0.001; “ChangeTrend”, Table 2), an effect that did not differ by group (p=0.35; “ChangeTrend×SUD”, Table 2). For Novel sets with changed S-R contingencies, control participants demonstrated steeper rates of performance improvement post-change relative to pre-change (p<0.001; “NewResponse×ChangeTrend”, Table 2). In contrast, the SUD group showed a shallower rate of improving performance for Novel changed sets post-change relative to pre-change. For Familiar unchanged sets, the control group demonstrated a steeper rate of improvement in the re-learning epoch relative to the pre-change epoch (p<0.001; “Set×ChangeTrend”, Table 2). During re-learning, the SUD group showed significantly shallower rates of performance improvement for unchanged Familiar S-R sets (p<0.05; “Set×ChangeTrend×SUD”, Table 2), while the interaction between SUD status, set type, contingency change, and the change in learning rate in the re-learning epoch was not significant (p=0.08; “Set×NewResponse×ChangeTrend×SUD”, Table 2). To summarize, S-R contingency change did not reveal a global impairment in response flexibility or inhibitory control among people with an SUD history. In fact, for the changed Novel S-R set, the SUD group's performance was less impaired than that of the control group immediately following contingency change, resulting in a more rapid performance recovery for the SUD group (Fig. 3, yellow dashed line).

Figure 3. Mean accuracy values during pre- and post-change task performance.

Solid lines represent the control group and dashed lines represent the SUD history group. Overall, a dropoff in performance occurred in blocks with changed response contingencies and performance improved over the course of re-learning (“Post” panels; Table 2); the rate of re-learning compared to the initial (“Pre”) learning rate was dependent upon group status, set-type, and change status (Table 2). (A) Panels depict performance in Familiar sets (magenta) during pre- and post-change. Dark blue lines in the right panel (“Post”) indicate performance in the set with unchanged response contingencies during the re-learning phase; performance in the response-changed set shown in magenta. (B) Performance in Novel sets (yellow) during pre- and post-change. Black lines in the right panel (“Post”) indicate performance in the set with unchanged response contingencies during re-learning; performance in the response-changed set shown in yellow. Performance dropped more dramatically after response change for Novel sets relative to Familiar sets (Table 2). Additionally, the control group showed steeper learning rates for Novel changed sets relative to the SUD group. Corresponding model predicted values are shown in panels C and D.

Responding to a stimulus with an action that is no longer valued (i.e. no longer positively reinforced) is taken as an indicator of habit-based, rather than goal-directed, responding. Thus, to quantify the habitual nature of responding after response contingency change, we evaluated the percentage of perseverative errors during the re-learning epoch. A one-way ANOVA between group for each set-type indicated significant differences between groups for the overall percentage of perseverative errors for the Familiar set-type (p=0.004), but not for the Novel set-type (p=0.43). These results reflect the fact that when trying to replace the well-established Familiar S-R associations, errors made by the SUD group were more apt to be perseverative errors (p=0.002; Fig. 4). No such group difference was observed for replacement of more recently established Novel S-R associations (p=0.146; Fig. 4). These findings indicate the more habitual nature of responding in the Familiar S-R sets among SUD participants.

Figure 4. Post-change percentage of perseverative errors by group for S-R sets.

The percentage of perseverative errors in the FAMILIAR set (magenta) with changed response contingencies significantly differed by group, with SUD history participants making incorrect responses that were perseverative in nature (F(3,58)=4.88, p=0.004). In contrast, the percentage of perseverative errors in the NOVEL set (yellow) with changed response contingencies did not differ between groups (F(3,58)=0.93, p=0.43). Error bars represent standard error of the mean.

To evaluate the contribution of abused substance type to perseverative responding during S-R re-learning, we stratified SUD participants into two categories: history of stimulant dependence (n=11), or no history of stimulant dependence (n=11). We found a significant difference in perseverative errors during Familiar S-R re-learning (Kruskal-Wallis test, p=0.009; Fig. 5, pink bars), but not during Novel S-R re-learning (p=0.182; Fig. 5, yellow bars). Post hoc tests (Bonferroni corrected, p<0.025) demonstrated that participants with or without a history of stimulant dependence made significantly more perseverative errors during re-learning of Familiar S-R sets relative to controls (stimulant history, p=0.007; no stimulant history, p=0.009). However, only the stimulant dependence group showed a trend toward more perseverative responding during Novel S-R re-learning relative to controls (stimulants; p=0.038; no stimulants; p=0.342). The results in the Novel condition suggest that stimulant addiction may be associated with an even more rapid transition to habitual responding.

Figure 5. Post-change percentage of perseverative errors by abused substance type group for S-R sets.

We categorized participants according to substance dependence history as follows: no history (control), no stimulant dependence (alcohol or opiate dependence; “No Stims”), or stimulant and alcohol dependence (stimulants; “Stims”). Overall nonparametric comparison of the three groups indicated a significant difference in the percentage of perseverative errors for the FAMILIAR set (magenta), χ²(3)=11.67, p=0.009. The groups did not differ in terms of the percentage of perseverative errors for the NOVEL set (yellow), χ²(3)=4.86, p=0.182. Post-hoc tests corrected for multiple comparisons (p=0.025) demonstrated that, compared to controls, participants with a history of stimulant dependence committed a higher percentage of perseverative errors for both the FAMILIAR set (z=2.69, p=0.007) and the NOVEL set (z=2.07, p=0.038). In contrast, participants with no history of stimulant dependence committed a higher percentage of perseverative errors when compared to controls only in the FAMILIAR set (z=2.60, p=0.009), not in the NOVEL set (z=2.69, p=0.342).

Discussion

We demonstrate that people with SUDs learn new S-R associations as well as control subjects do and can flexibly adapt newly learned S-R associations, but are specifically impaired in overcoming well-learned S-R associations. Notably, those with SUDs differ from controls only in terms of perseverative errors committed during well-established S-R replacement, indicating the more habit-based nature of their responses. These findings suggest that people with a history of an SUD more rapidly acquire habitual responding outside of the drug-taking domain.

Prior Studies Linking Habit and Addiction

Despite extensive investigation of drugs of abuse and habit in animal models, modest translation of these experimental paradigms to human studies has occurred to date. Young, light-smoking adults will pursue both cigarette and chocolate rewards via goal-directed strategies (Hogarth & Chase, 2011). Nicotine dependence was low in this sample; however, (Hogarth, Chase, et al., 2012) made similar findings in a sample of daily and non-daily smokers, in addition to finding a positive correlation between motor impulsiveness and habitual responding. These studies suggest that habitual drug consumption may associate with personality factors that predispose individuals toward habit-based responding.

Hogarth and colleagues have also found that acute alcohol intake renders the selection strategy for both water and chocolate rewards habitual (Hogarth, Attwood, Bate, & Munafo, 2012), which is consistent with data showing that exposure to alcohol potentiates habitual responding in rats (Corbit et al., 2012). A recent fMRI study of alcohol dependent patients in an instrumental choice task (S De Wit, Niry, Wariyar, Aitken, & Dickinson, 2007) found evidence of preferential S-R based responding, rather than goal-directed actions, in alcohol dependent individuals relative to controls (Sjoerds et al., 2013). Furthermore, relative to controls, the alcohol dependent group increased activation of the posterior putamen and reduced vmPFC activation during instrumental choice. Although these data are consistent with the animal literature associating chronic exposure to drugs of abuse with an over-reliance on S-R response strategies, many of the alcohol dependent patients in Sjoerds et al.'s study were concurrently using psychoactive medications for depression and anxiety disorders, precluding unequivocal attribution of group differences solely to alcohol dependence. Regardless, these results point to neural correlates within frontostriatal circuits for enhanced reliance on S-R versus goal-directed actions. Such findings enable strong predictions about expected differences between people with SUDs and healthy controls in terms of neural activation associated with response selection in the HABIT.

Neurobiology of habit in humans

Although the neural bases of behavioral differences in S-R learning among people with SUDs is largely unexplored, the SUD neuroimaging literature suggests that alterations in frontostriatal circuit recruitment underlie atypical behavior in SUDs (Ersche et al., 2012; Goldstein & Volkow, 2011; Kalivas, 2008; Konova et al., 2012; Koob & Volkow, 2010; Olausson et al., 2007; Park et al., 2010). Although this work has not investigated habits per se, it logically follows that impaired frontal control of striatal output signals could yield over-reliance on striatal habit circuits. Drugs of abuse may alter frontostriatal circuitry (Izquierdo & Jentsch, 2012) such that frontal input to the striatum can no longer effectively act as a ‘switch’ to regulate the contribution of brain signals for automatic, habitual responding versus goal-directed action (K. S. Smith & Graybiel, 2013; Kyle S. Smith, Virkud, Deisseroth, & Graybiel, 2012).

A unique aspect of the HABIT paradigm is the ability to measure behavior during attempts to overcome habitual responding; using perseverative errors as an index of habit-based responding and continuing task conditions that require goal-directed responding allows us to measure the flexibility of behavior over an extended period after response contingency change. Essentially, we are able to measure the ability to ‘break’ habits that have been formed within this task. The ability to change or ‘break’ habitual behaviors has not directly been tested in humans to our knowledge, but converging evidence from both recent animal and human studies relate drugs of abuse and frontostriatal circuitry to the regulation of behavioral change. In primates, prolonged cocaine intake profoundly impairs S-R re-learning (Jentsch, Olausson, De la Garza, & Taylor, 2002). These data suggest that chronic drug exposure potentiates habitual response selection and further supports a role for extended substance abuse in altering the circuits underlying S-R learning and replacement. In rodents, optogenetic perturbation of the infralimbic portion of the mPFC results in a switch from a previously to recently learned behavior, and thus facilitates the replacement of habitual behaviors (Kyle S. Smith et al., 2012). Computational modeling of human choice behavior in which prefrontal brain regions ‘arbitrate’ between habit-based or goal-directed responses further supports these animal findings (Lee, Shimojo, & O'Doherty, 2014). Transcranial magnetic stimulation applied to the DLPFC shifts the balance between goal-directed versus habit-based response selection strategies (Knoch, Brugger, & Regard, 2005; Smittenaar, FitzGerald, Romei, Wright, & Dolan, 2013). Taken together, these studies provide compelling evidence for the regulation of behavioral control via frontostriatal circuitry. An important future direction is to determine whether abnormal functioning of these same frontostriatal circuits underlies the atypical S-R learning and replacement we find in people with a history of addiction.

It is important to note that elevated perseverative errors in the changed Familiar S-R sets in the SUD group is unlikely to reflect impaired response inhibition, as inhibitory impairments should have manifest in the Novel condition as well as the Familiar condition, based on nearly equal performance in the Novel condition prior to the contingency change, particularly among the SUD group. However, we only observed this deficit in the highly practiced Familiar condition in which the SUD group appears to have transitioned to a more automatic S-R strategy. One could make the case that suppressing a more automatized action requires greater response inhibition, and that only under this higher “inhibitory load” condition did a deficit in the SUD group emerge; however, this argument merely lends support to our interpretation of a more rapid transition to an automatic response strategy in people with an SUD history.

Possible Role of Stress in the SUD Group Findings

While we make the case above that atypical frontostriatal function likely underlies the apparent earlier switch to dominance of habit-based responding in the SUD group, a growing body of literature shows that stress can potentiate habit-based responding in both humans (Schwabe et al., 2007; Schwabe & Wolf, 2010, 2011, 2013) and animal models (Dias-Ferreira et al., 2009). This tendency for stress to shift the balance of behavior from goal-directed to habitual has been investigated pharmacologically in humans, with evidence indicating roles for both elevated cortisol levels and increased noradrenergic activity (Schwabe, Hoffken, Tegenthoff, & Wolf, 2011; Schwabe, Tegenthoff, Hoffken, & Wolf, 2012). Moreover, neuroimaging research has found that participants subject to chronic psychosocial stress fail to change responses after outcome devaluation, indicative of habit-based actions (Soares et al., 2012). The stressed participants in that study showed greater activation of the putamen during response selection after devaluation, relative to non-stressed controls, consistent with prior links between the putamen and habitual responding (S. de Wit et al., 2009; Tricomi et al., 2009). Notably, the behavioral and neural effects of stress in the Soares study were reversible, declining after the stressful period ended; this demonstrates the plasticity of the neural systems regulating habitual actions, and holds promise for interventions to facilitate behavioral change of ingrained behaviors.

The evidence that stress can promote habit-based responding, together with evidence of dysregulated hypothalamic-pituitary-axis (HPA) function in individuals with SUDs (Anthenelli, Maxwell, Geracioti, & Hauger, 2001; King et al., 2002; Kreek, Nielsen, Butelman, & LaForge, 2005; Lijffijt, Hu, & Swann, 2014; Porcu, O'Buckley, Leslie Morrow, & Adinoff, 2008), suggest that the behavioral differences that we observed in the SUD group could reflect greater stress levels in the SUD group. We did not collect physiological or subjective report measures of stress for this study, although we did collect measures of anxiety; the groups did not differ in terms of state anxiety, but the SUD group did report slightly higher levels of trait anxiety (Table 1), suggesting possibly higher levels of chronic stress in the SUD group compared to controls. Stress is well known to precipitate relapse (Sinha, 2012), and although the underlying mechanisms are not well understood, it is tempting to speculate that a contributing factor could be stress-induced promotion of habitual responding in people with SUDs. This question can be addressed with the HABIT paradigm, which may ultimately identify new therapeutic approaches to relapse prevention.

Study Limitations

The observation of more habit-based responding in the SUD group could be a consequence of chronic drug exposure, or a predisposing trait that contributes to SUD vulnerability; these alternatives cannot be disentangled by the current study. If this heightened propensity to establish habits in people with SUDs predates the SUD, it would represent a promising, unexplored intermediate phenotype for SUDs. A further limitation is the SUD sample studied. Participants were recruited based on any lifetime history of an SUD (including an alcohol use disorder), which yielded a heterogeneous population. Given the distinct effects of differing abused substances on neurotransmitter systems, it is rather unlikely that the behavioral effects we observed reflect common neural dysfunction caused by chronic substance abuse. However, biological predispositions play a large role in SUDs and that heritability is not necessarily substance-specific (Hicks, Iacono, & McGue, 2012). As such, one would expect to find common neural substrates across different substance abuse categories underlying shared behavioral deficits that represent pre-existing vulnerability factors. Our finding here of propensity to more rapidly transition to habit-based responding could theoretically contribute to establishing and/or maintaining compulsive, habitual substance use and as such, could theoretically play a role as a pre-existing risk trait. Be that as it may, this heterogeneity, coupled with our small sample size, precludes drawing conclusions regarding specific substances or poly-substance use. The range of disease severity was also limited in our sample, with all participants falling at the severe end (range: 6-11) (Hasin et al., 2013); thus, we were unable to assess whether SUD severity correlates with greater propensity for habitual responding. Another limitation was our exclusion of individuals currently using any psychoactive medications, or with co-morbid mental health disorders, neurological conditions, or below normal IQ. The advantage of this “clean” sample is our confidence in attributing group differences in behavior to SUD history, but among the SUD population at large, co-morbidities and psychoactive medication use is common. These exclusion criteria also likely and substantially increased our power to detect group effects, as different co-morbid conditions may have either amplified or compensated for excess habitual responding; psychoactive medications may have similarly increased variance. Finally, our participants with SUDs were abstinent from substance use, and as such, their engagement of motivational circuitry and the ability to form habitual associations might be substantially different from people in the active phase of an SUD. These limitations point to key future avenues of research that will expand the scope of our understanding of habit-based responding in addiction.