Perhaps More Consideration of Pavlovian–Operant Interaction May Improve the Clinical Efficacy of Behaviorally Based Drug Treatment Programs

Abstract

Drug abuse remains costly. Drug-related cues can evoke cue-reactivity and craving, contributing to relapse. The Pavlovian extinction-based cue-exposure therapy (CET) has not been very successful in treating drug abuse. A functional operant analysis of complex rituals involved in CET is outlined and reinterpreted as an operant heterogeneous chain maintained by observing responses, conditioned reinforcers, and discriminative stimuli. It is further noted that operant functions are not predicated on Pavlovian processes but can be influenced by them in contributing to relapse; several empirical studies from the animal and human literature highlight this view. Cue-reactivity evoked by Pavlovian processes is conceptualized as an operant establishing/motivating operation. CET may be more effective in incorporating an operant-based approach that takes into account the complexity of Pavlovian–operant interaction. Extinction of the operant chain coupled with the shaping of alternative behaviors is proposed as an integrated therapy. It is proposed that operant-based drug abuse treatments (contingency management, voucher programs, and the therapeutic work environment) might consider incorporating cue-reactivity, as establishing/motivating operations, to increase long-term success—a hybrid approach based on Pavlovian–operant interaction.

Overview

Drug and alcohol abuse remains costly to society. In 1992 alone, the total cost of drug and alcohol abuse was $246 billion (Harwood, Reiter, Kleiman, Kopp, & Cohen, 1999. Assuming a 12.5% increase due to inflation and cost of living every 3 years, based on these estimates, the total cost of drug abuse in 2012 may have been approximately $500 billion (see also Kelly & Yeterian, 2012). Drug abuse, its treatment and prevention, impacts segments of the economy (legal, medical, health, insurance, pharmaceutical, education, lost time at work, and identity theft). The cost to conduct the research summarized in this paper alone is unimaginable.

This paper discusses drug cue-reactivity, one historically popular behavioral drug abuse treatment approach (cue-exposure therapy [CET]), and suggests that operant drug treatment approaches should incorporate aspects thereof. The literature regarding CET's lack of clinical efficacy is cited concerning spontaneous recovery, context renewal, reinstatement, and general failure to carry out complete extinction of operant drug-taking rituals (Conklin & Tiffany, 2002). Drug-seeking behavior and drug-taking rituals are proposed here as behaviors embedded within a heterogeneous operant chain, involving observing behavior, discriminative stimuli, and operant conditioned reinforcers, all of which can be modulated by establishing/motivating operations. Brief descriptions and characterizations of these operant functions, rather than comprehensive reviews of the immense literature on each topic, are presented. The final section briefly summarizes some of the more recent operant-based drug abuse treatment programs (e.g., voucher programs, contingency management, therapeutic work environment) that appear to be clinically effective for short-term drug abuse treatment. However, the same problems posed for CET may also be limitations for these operant-based drug abuse treatment programs. Behavioral resurgence and behavioral momentum of drug-seeking and drug-taking behavior in addition to establishing/motivating operations (i.e., cue-reactivity) may be additional obstacles for long-term efficacy of operant-based voucher, contingency management, and therapeutic work environment programs. It is proposed that these operant approaches to drug abuse treatment may consider including CET to evoke cue-reactivity, particularly if methodologically couched as establishing operations for alternative non-drug–related behavior.

Cue-Reactivity and Cue-Exposure Therapy

History

Pavlov (1927) first suggested that drugs can function as biologically relevant unconditional stimuli (USs). In 1948, Wickler reported that opiate addicts exhibited withdrawal symptoms when exposed to environmental cues previously related to opiate abuse (i.e., cue-reactivity); he suggested that they were mediated by associative learning factors. Other reports from Wickler's lab followed, demonstrating associative processes related to conditioned opiate withdrawal (Wickler, 1971, 1973; Wickler & Prescor, 1967). Twenty-nine years later, Siegel (1975) demonstrated that drug tolerance, withdrawal, and “overdose” reactions could be evoked by Pavlovian (i.e., respondent) conditioning using rats. Briefly, administration of morphine elicits a series of Pavlovian unconditioned responses (URs; e.g., analgesia, hypothermia) in a distinct environmental context, which is conceptualized as the conditional stimulus (CS). Over consecutive sessions, tolerance develops to these URs within the drug-associated context CS but not in the context that was un-paired with morphine. Three common observations have been reported. First, the drug-context CS promotes the development of tolerance to the effects of morphine. Analgesia and core body temperature return to baseline levels when rats are tested under the drug state in the drug context CS. Second, when morphine is administered in a non-drug context, tolerance is reversed. Rats exhibit large morphine effects with substantial analgesia and hypothermia. Administration of doses higher than the initial training dose in the non-drug context evokes an “overdose” response. Finally, when the vehicle is administered in the drug-associated CS context, withdrawal-like responses occur. Rats exhibit hyperalgesia and hyperthermia. This last finding supports the conditioned compensatory response hypothesis (“drug opposite responses”) postulated to be the mechanism for the basis of conditioned (respondent) tolerance (Siegel & Ramos, 2002) that contributes to drug cue-reactivity discussed in the next section.

It should be noted that there are alternative associative views to Siegel's model (cf. Eikelboom & Stewart, 1982; Tiffany & Baker, 1981). In fact, the conditioned compensatory response is not readily observable with psychostimulants; priming with sedatives is required (for a detailed review, see Goudie & Demellweek, 1986). Thus, the conditioned compensatory response is not symmetrically elicited by CSs associated with specific US effects of sedatives and stimulants. In view of these asymmetrical response systems, (1) withdrawal and “craving” evoked by CNS sedatives versus stimulants cannot be equivalent and (2) cue-reactivity must differ as a function of drug class and drug of choice. Wickler's and Siegel's findings along with traditional behaviorally based treatment strategies (counterconditioning; e.g., Foa & Goldstein, 1978; cf. Wolpe & Plaud 1997) have been the fulcrum for understanding drug cue-reactivity and CET.

Common Cues That Evoke Drug Cue-Reactivity

Cue-reactivity is defined here as series of responses (physiological and/or the subjective verbal report of drug “craving”) that can be evoked by drug-related stimuli (e.g., Childress, Ehrman, Rohsenow, Robbins, & O'Brien, 1992; Childress, Mozley, McElgin, Fitzgerald, Reivich, & O'Brien, 1999; Ehrman, Robbins, Ternes, O'Brien, & McLellan, 1992; Garland, Franken, & Howard, 2012). It has been suggested that such stimuli (cues) sensitize behavior by repeated exposure to a drug or other primary natural reward (food, water, sexual activity) and promote “incentive motivation” (Robinson & Berridge, 1993, 2003; Troisi & Akins, 2004). Cue-reactivity can be modulated by drug withdrawal, stressors, emotions, or cues that elicit a conditionedcompensatory response as just discussed. A variety of drug-related stimuli evoke drug cue-reactivity and self-reported increases in drug “craving” in humans, all of which can evoke relapse (Carter& Tiffany, 1999b). For instance, EEG activity changes are modulated by exposure to cocaine-related cues (Horrell, El-Bath, Baruth, Tasman, Stewart, & Sokhadze, 2010). Similarly, marijuana-related pictures and paraphernalia evoke changes in P300 wave activity and increases in self-reported craving in abstinent cannabis users (Nickerson et al., 2010). Stress-related stimuli also increase ratings of self-reported drug craving for alcohol and cocaine (Preston & Epstein, 2011; Sinha, Fuse, Aubin, &O'Malley, 2000). In methadone-maintained participants, drug cue–related stimuli increase self-reported cravings and stress responses (Walter et al., 2008). Moreover, negative affective states also increase drug cue-reactivity (Stasiewicz et al., 1997), and the interaction between mood states and exteroceptive social cues has also been entertained within this clinical context (Laberg, 1990). Finally, interoceptive emotional/affective states (Field, Munafò, & Franken, 2009; Tiffany, 2009) and stressors (e.g., Sinha, 2001) modulate drug craving (Childress, Ehrman, McLellan, & MacRae, 1994) and self-administration in humans (e.g., Hodgins, el-Guebaly, & Armstrong, 1995). Positive emotions have also been considered to play a role in cue-reactivity (Niaura et al., 1988). A more recent investigation reported that alcohol-related visual stimuli elicited changes in heart rate activity in alcoholics, which predicted relapse following treatment (Garland, Franken, & Howard, 2012). Thus, there is reasonable evidence that Pavlovian stimuli (i.e., “incentive cues”) evoke physiological responses that play a role in cue-reactivity and subsequently modulate drug-reinforced behavior, thereby promoting relapse (see also Hogarth, Dickinson, & Duka, 2010). Of course, cue-reactivity involves a host of interoceptive stimuli within the organism or recovering drug abuser. Multiple sensory stimuli (interoceptive and exteroceptive) likely interact as a gestalt modulating drug craving, drug seeking, drug taking, and relapse. Cue-reactivity will be conceptualized as operant establishing/motivating operations in a later section.

The Cue-Exposure Therapy (CET) Paradigm

CET for substance abuse employs respondent (Pavlovian) extinction trials. The primary objective of CET is to desensitize drug cue-reactivity (i.e., “craving”) to minimize the likelihood of relapse in recovering drug abusers (e.g., Childress, McLellan, & O'Brien, 1986a, 1986b; McLellan, Childress, Ehrman, & O'Brien, 1986; O'Brien, Childress, Mclellan, & Ehrman, 1990; Rohsenow, Childress, Monti, Niaura, & Abrams, 1991). It is beyond the scope of the present paper to review all studies conducted on CET. The objective here is to note a cross-section of this literature to provide the reader with an overview and breadth of the literature.

Clinically, drug abusers are exposed to pre-drug– and drug-related stimuli (including drug paraphernalia) in experimentally simulated drug-taking contexts. Participants engage in a “cook up” session or prepare syringes complete with a tied tourniquet (Childress et al., 1986a, 1986b; McLellan et al., 1986; O'Brien et al., 1990). These exteroceptive contextual cues, along with target behaviors in the drug-taking sequence, are conceptualized as Pavlovian CSs (see Siegel & Ramos, 2002). Video footages of a “cook up” session may also be shown to the client prior to engaging in the ritual(s). Various psychophysiological measures, including heart rate, galvanic skin response, and body temperature, along with self-reported ratings of “craving” or “crash” are used as dependent measures that have been shown to dissipate throughout extinction. These measures are monitored throughout the drug-taking ritual, which includes all behaviors leading up to the actual self-administration response. Most critically, however, participants do not self-administer the drug US—nor do they self-administer a placebo. Therefore, the final activity most proximal to the drug effect (the US or primary operant reinforcer) does not undergo extinction (see the next section). Actual voluntary behaviors in the drug-taking ritual are conceptualized as self-administration cues that are conceptualized to function as Pavlovian CSs (Siegel & Ramos, 2002. Furthermore, CET targets stimuli and responses that are most proximal to self-administration of the drug, rather than those more distal that occur long before actual drug use (Conklin, Robin, Perkins, Salkeld, & McClernon, 2008; Siegel & Romos 2002).

Historically, CET has been used to treat drug craving and self-administration of several drugs of abuse, including cocaine (Ehrman, Robbins, Childress, & O'Brien, 1992; Kilts et al., 2001), heroin (Fan et al., 2009; Marissen, Franken, Blanken, van den Brink, & Hendriks, 2005), alcohol (Loeber, Croissant, Heinz, Mann, & Flor, 2006), and tobacco (García-Rodríguez, Pericot-Valverde, Gutiérrez-Maldonado, Ferrer-García, & Secades-Villa, 2012; Niaura et al., 1999).

Challenges and Caveats for CET: Spontaneous Recovery, Reinstatement, Renewal, and Incomplete Extinction

Despite its popularity over the last 25 years or so, the clinical success of CET has been equivocal (Conklin & Tiffany, 2002). Meta-analyses and other more direct empirical evaluations (Carter & Tiffany, 1999a, 1999b; Marissen, Franken, Blanken, van den Brink, & Hendriks, 2007) have revealed weak reliability (see also Drummond, 2001) with little or no long-term follow-up data beyond 6–12 months (see Conklin & Tiffany, 2002, p. 158). For example, in one study published after 2002, Marissen et al. (2007) reported a high dropout rate for participants in CET for opiate abuse.

Spontaneous recovery, reinstatement, and renewal

Why has CET been so unsuccessful? In two separate theoretical reviews from the animal learning literature, Bouton and Schwartzentruber (1991) and, later, Conklin and Tiffany (2002) argued convincingly that mere presentation of drug-related cues (CSs) to a drug abuser (Pavlovian extinction) may be insufficient to prevent drug abuse relapse. As known by many who study it, extinction is only a temporary suppression (or inhibition) of behavior, rather than an un-learning of the original CS → US relationship (Bouton, 2002; Rescorla, 2004; Robbins, 1990).Accordingly, both of these reports outlined several caveats for CET based on phenomena commonly observed in the animal learning literature (see also Bouton, 2002): Spontaneous recovery is the return of the CR following a delay period after extinction. As Conklin and Tiffany (2002) noted, the number of extinction sessions carried out in the investigations they summarized varied from only 1 to 9 sessions. Pavlov (1927) first demonstrated that spontaneous recovery increases with the passage of time after the last extinction trial (or session) and is proportional to the amount of extinction sessions carried out. Drug abusers are likely to be more vulnerable to relapse upon encountering drug-related cues long after extinction, especially if only a few extinction trials are carried out. Spontaneous recovery of operant responding also increases with the passage of time (Rescorla, 1997). Reinstatement evokes a return of conditioned responding by the mere presentation of the US or primary operant reinforcer during or following extinction. This phenomenon has been reported in the Pavlovian literature (e.g., Rescorla & Heth, 1975) and in the operant literature with drug reinforcement (Leri & Stewart, 2001; Shaham, Shalev, Lu, de Wit, & Stewart, 2003). An excellent review that summarizes the validity of reinstatement in relapse was composed by Epstein, Preston, Stewart, and Shaham (2006). Non-contingent presentation of a drug reinforcer following extinction promotes significant recovery of a self-administration response. This reinstatement effect has also been demonstrated to come under discriminative control with interoceptive drug states for food reinforcement (Troisi, 2003b). Thus, for example, if a recovering opiate abuser is medically prescribed morphine or some other opiate for pain management, or if a recovering alcoholic is prescribed a benzodiazepine for anxiety, the probability of relapse may increase in both instances (cf. Posternak & Mueller, 2001). Finally, context renewal is a return of the CR elicited by the CS following extinction in a different environmental setting (Bouton, García-Gutiérrez, Zilski, & Moody,2006; Bouton & Ricker, 1994; Thewissen Snijders, Havermas, van der Hout, & Jansen, 2006; see also Nakajima & Tanaka, 2000, for an operant example). CET typically is carried out in a medical or laboratory setting, but re-exposure to drug cues outside of this setting may also increase the likelihood of relapse. In fact, Bouton et al. (2006) have demonstrated that extinction in multiple contexts does not necessarily minimize recovery of responding.

Incomplete extinction

CET's long-term efficacy has been poor because it is predicated solely on Pavlovian extinction trials and likely renders the recovering addict vulnerable to spontaneous recovery, reinstatement, and renewal. Furthermore, as noted earlier, CET prohibits the drug abuser from engaging in the final self-administration response and consequently does not extinguish operant behavior that directly produces the reinforcing drug effect (Conklin & Tiffany, 2002, pp. 163, 165). By analogy, Conklin and Tiffany (2002, p. 163) contrasted Pavlovian CET to a rodent intravenous drug self-administration model in which lever pressing is initially maintained by IV drug infusion. During Pavlovian extinction, the lever is removed from the chamber and the animal is simply exposed to the conditioning context and specific CSs that were previously paired with the drug US. Such conditions are not favorable for operant extinction,, that is, mere exposure to the training context without also providing the opportunity to engage inthe operant response. At a later point, protraction (or reinstallation) of the lever, a visual cue in and of itself, wouldrenew extinguished stimulus control by the context (Troisi, 2011; Troisi, LeMay, & Järbe, 2010). The Pavlovian conditioning literature has reliably demonstrated this with more hierarchical stimuli that modulate the role of binary CS → US relationships, occasion setters (Holland, 1983; Rescorla, 1985, 1986a, 1986b, 1987). From a Pavlovian perspective, all elicited respondents and the cues that evoked them would be predicted to undergo extinction by removing the response manipulandum (the lever), except for stimuli most proximal to the primary reinforcer (i.e., the lever itself). When the lever is reinserted, the visual stimulus configuration would be expected to renew the context–lever relationship. From an operant perspective, the reinsertion of the lever would function as anS^D, responding under which would have been spared extinction. Consequently, the visual presence of the lever would continue to set the occasion for the operant target response (lever pressing), which would also have been spared extinction. Hence, “relapse” would be evident. Two rather nicely summarized presentations of several behavioral phenomena involved in relapse were recently published by Thrailkill & Shahan (2012) and Bouton, Winterbauer, & Vurbic (2012).

A Suggested Improvement for CET

In addition to increasing the number of extinction sessions or trials, Conklin and Tiffany (2002) proposed that extinction of the operant response within the operant three-term contingency could be considered a more effective strategy for drug abuse treatment research—S^D: R → S^r+ (the drug-taking context, the drug taking act → the reinforcing drug properties). They also noted the importance of the operant occasion-setting function of the S^D within the drug-seeking and -taking rituals. To quote Conklin and Tiffany (2002),

Breaking the association between drugs and reinforcing drug effects remains a challenge to addiction treatment. Therefore, unreinforced drug administration might prove to be cue exposure treatment's greatest asset. (p. 165)

This statement implies that CET involving Pavlovian CSs may functionally differ from operant S^Ds and operant conditioned reinforcers. A study by Krank and Wall (1990) illustrated this very point. In that study, a sweetened EtOH solution was contingent on rats' lever-press responses in the presence of a discriminative stimulus, but not in its absence. Extinction of responding for sweet water (without EtOH) under discriminative control by a light was more complete, and more resistant to recovery, than it was in the absence of the light; however, the taste alone also maintained responding throughout extinction. Although the study was not statistically robust, it was theoretically interesting. It would be informative to replicate the Krank and Wall study with intravenous administration of cocaine or morphine. It was unclear whether the ethanol alone, and not the taste, maintained responding (e.g., Katner & Weiss, 1999). A replication might arrange for drug self-administration to come under discriminative control by a light (or a tone). Completion of the schedule of reinforcement would result in the pairing of another cue with the IV drug; then extinction could be carried out. One group would receive extinction under the S^D that did not lead up to the proximal cue; the other group would have no scheduled outcome. This proposed study awaits research attention.

Conklin and Tiffany (2002) suggested that a better approach to drug abuse treatment might be to allow the drug user to self-administer an IV placebo, smoke a denicontinized cigarette, or drink a non-alcoholic beer in alcohol-related contexts (e.g., Blakey & Baker, 1980; but see Nace, 2005). Functionally speaking, such an operant approach would provide the opportunity for extinction of all three components within the three-term operant contingency (S^D: R; R → S^r+; and S^D → S^r+). Of course, there is a vast animal literature on extinction of IV drug–maintained responding (e.g., Carrol & Comer, 1996); Leri & Stewart, 2001; and see Taylor, Olaussson, Quinn & Torregrossa, 2009, for a cognitive neuroscience view of extinction of drug related cues and behavior). In the process of extinguishing the operant discrimination, any respondent (CS → US) relationships that may have accrued to the S^D and the response manipulandum (the lever) would also extinguish. After all, why would one expect that Pavlovian extinction alone would eliminate operant behavior?

An Operant Analysis of the Drug-Taking Ritual in CET: The Heterogeneous Operant Chain

A closer functional analysis of complex drug-taking rituals reveals a series of operant responses that are occasioned by operant discriminative stimuli (S^Ds), conditioned reinforcers, and the responses for which they set the occasion, rather than contiguous pairings among stimuli (CSs) that occur in time. A Pavlovian interpretation of the drug-taking behavior might suggest the development of higher order conditioning (Holland & Rescorla, 1975; Rescorla & Cunningham, 1979) among each of the sets of stimuli within the drug-taking ritual that are correlated with the terminal drug US. In one form of higher order conditioning (i.e., second-order conditioning), a neutral stimulus acquires CS₂ functions due to its pairing with an established CS₁. Higher order conditioning is difficult to attain beyond that of second-order stimulus control (Gewirtz & Davis, 2000). In contrast, a heterogeneous operant chain can arrange numerous linkages with discriminative stimuli and conditioned reinforcers (D'Andrea, 1969; Lattal & Crawford-Godbey, 1985). For example, in a rodent model, the presence of white noise might function as an operant S^D₃ that sets the occasion for a chain-pull response (R₃). The chain-pull results in the presentation of a tone (S^D₂), which sets the occasion for a nose poke (R₂). Nose poking then produces a light (S^D₁), which then sets the occasion for lever pressing (R₁) to be reinforced by the primary food reinforcer (S^r+). Hence, the sequence is diagrammed, S^D₃: R₃ → S^D₂: R₂ → S^D₁: R₁ → S^r+. In this example, the light and the tone not only function as discriminative stimuli but also as conditioned reinforcers that are contingent on chain pulling and nose poking, respectively (Lattal & Crawford-Godbey, 1985; Zimmerman, 1959). Simple heterogeneous and homogeneous response chains have been demonstrated with drug reinforcement (Pelloux, Everitt, & Dickinson, 2007). As noted by Lattal and Crawford-Godbey (1985), complex and extended behavioral repertoires can also be established in rats with this methodology (see Pierrel & Sherman 1963). The heterogeneous chain should be more often applied to the study of drug abuse. For instance, Lu, Li, Hou, Chen, Chi, and Liu (2010) demonstrated that intravenous heroin maintained responding on a one-link chain with two topographically different responses in rats. In that study, nose poking opened up a door allowing the rat to traverse an alley that was followed by drug delivery. In this example, the open door reinforced nose pokes but also functioned as an S^D in occasioning alley crossing.

From an operant perspective, the rituals used in CET represent rather long, complex heterogeneous chains involving topographically specific responses that are reinforced by consequential exteroceptive stimuli that function as operant conditioned reinforcers—not Pavlovian CSs. The final response (self-administration) most proximal to the primary reinforcer (the drug effect) is not accessible to the drug abuser in a Pavlovian CET regimen as noted above. Thus, without extinction of the final response, the chain leading to it would survive with the passage of time (see Kuhn, Lerman, Vorndran, & Addison, 2006).

Extinction of the heterogeneous operant chain has been carried out in a clinical population to modify disruptive behavior in children (Kuhn et al., 2006). That report showed that extinction of the entire sequence was more effective than only extinguishing one response. The application of Conklin and Tiffany's (2002) extinction suggestion to an operant heterogeneous chain analysis of a drug-taking ritual might be carried out as follows: R₅ (drug seeking behavior) → S^D₄ (shooting gallery context): R₄ (arrange drug paraphernalia) → S^D₃ (tied tourniquet): R₃ (look for syringe) → S^D₂ (presence of syringe): R₂ (pick up syringe) → S^D₁ (holding syringe): R₁ (pierce skin and push plunger) → placebo (no drug effect). R5 might involve observing responses (which will be described in a later section) that are reinforced by the presence of other conditioned reinforcing S^Ds in effect at the time. To be sure, this chain could likely to be much longer and have many other S^Ds and responses embedded therein. Extinction procedures could be extended beyond the fourth S^D in an attempt to extinguish drug seeking (observing responses). Furthermore, if a booting ritual occurs following self-administration, it could be extinguished as well (Note: Booting involves pulling the plunger back, drawing blood out of the vein, and re-injecting multiple times; McElrath, 2006). Booting may be accounted for by the conditioned reinforcing effects of the discriminative stimuli involved in this ritual.

Theoretically, in the laboratory setting, it would be of interest to establish a long heterogeneous operant chain that is reinforced by drug administration (e.g., Lu et al., 2010). Such a chain could involve an observing response and several S^D links (more than just one) with topographically distinct operant responses (white noise: chain pull → tone: nose poke → light: lever press → drug infusion). In this manner, each exteroceptive S^D could be examined individually for its operant conditioned reinforcing effects by allowing each to function as a consequential outcome for a novel operant response (cf. Shahan, 2002). Next, each S^D could be presented alone in a standard Pavlovian extinction procedure (i.e., CET) versus an operant extinction procedure in which the reinforcer is omitted but the response chain and stimuli within it remain in effect (e.g., D'Andrea, 1969). The long-term efficacy could be determined by tests for spontaneous recovery of each response to the specific S^Ds. Empirically, which extinction procedure would yield the least spontaneous recovery—the Pavlovian respondent method, or the operant method? This proposition may develop a better animal analog of CET.

The tangible accessibility of the final response manipulandum in the drug-taking chain may certainly function as a conditioned reinforcer (an “incentive stimulus”) for discriminative stimuli and responses that lead up it—those most distal from self-administration act (see Conklin et al., 2008). However, operant conditioned reinforcement is not predicated on, or reducible to, basic Pavlovian CS → US relationships (Dinsmoor, 1983, 1985; Fantino, Preston, & Dunn, 1993; Fantino & Romanowich, 2007; Rescorla, 1990, 1992, 1994; cf. Shahan, 2002); therefore, non-reinforcement of binary Pavlovian relationships by mere exposure to the CS alone would not undermine the conditioned reinforcing functions of the stimuli most distal to the drug effect, and even those most proximal to the drug reinforcer (see Conklin et al., 2008). Yet, CET and assessments of cue-reactivity continue to fail in acknowledging these most critical operant–respondent distinctions. From an operant perspective, Pavlovian relationships are neither necessary nor sufficient for sustaining the complex drug-taking rituals. It is only through “voluntary” operant responses that specific CS → US relationships are likely to emerge as a result of the drug-taking rituals of humans. As “incentive cues” in drug abuse, Pavlovian–respondent relations are likely to be artifacts of operant functions rather than causal mediators of response–reinforcer relationships (R → S^r+). This hypothesis may be especially true when considering that relapse does occur without cue-reactivity, and cue-reactivity does not always culminate in relapse (Drummond, 2001; Niaura et al., 1988; Rohsenhow & Monti, 1999; Tiffany & Carter, 1998; Tracy, 1994). If “craving” (i.e., cue-reactivity) is evoked by Pavlovian processes, and craving is not always correlated with relapse, then a Pavlovian interpretation alone cannot account for relapse and cue-reactivity. This may be another reason why CET has been so ineffective over the years.

The Complexity of Drug-Related Cues Responses, and the Drug-Taking Act

By now it is more than evident that drug seeking and drug self-administration is operant, not respondent, elicited reflexive, behavior. Drugs of abuse are positive and negative reinforcers (Everitt & Robbins, 2005; Winger, Woods, Galuska, & Wade-Galuska, 2005). Environmental events previously associated with drugs not only evoke physiological changes as the result of Pavlovian processes but also set the occasion (Skinner, 1938) for drug use and abuse. Several hypothetical examples follow.

A light that reads “Open” posted outside of a bar, club, or pub signals (i.e., sets the occasion for) the availability of alcohol, which then sets the occasion for a series of complex operant responses, including pulling up, walking in, sitting down at the bar, ordering a draft from the bartender, and consuming the draft. This sequence is then followed by the pharmacokinetics and dynamics of the alcohol effect (the reinforcer). If we further assume that an individual is “hot,” “thirsty,” “angry,” or even celebratory, the responsiveness to the “Open” sign may be greater. These interoceptive responses may be establishing operations, as discussed in a later section concerning establishing operations. In this hypothetical example, the “Open” sign functions as an operant discriminative stimulus—an S^D; it always predicts alcohol availability, but it is only correlated with alcohol if, and only if, the series of operant responses follow, which is best conceptualized as an operant chain, as discussed above. All of the responses that are occasioned by the “Open” sign are also maintained by other environmental stimuli (e.g., view of an empty bar chair, the bartender's request).

In contrast, passing by a cocaine “crack house” or heroin “shooting gallery” may only intermittently set the occasion for availability of the drug of choice. In this example, there is perhaps far less “certainty” in terms of the drug availability. By comparison, holding a large sum of money (i.e., a generalized operant reinforcer for other behaviors), which can evoke cocaine craving (Garavan et al., 2000), may provide the basis to procure a drug of choice but does not predict drug availability in the way that the “Open” sign does for alcohol. The presence of drug abusing peers may function similar to passing a crack house or shooting gallery.

In the examples above, the “Open” sign, the crack house or shooting gallery, and the large sum of money are perhaps best conceptualized as discriminative stimuli that are most distal (most remote) from the actual drug effect. Of course, there are also cues more proximal to the drug US/reinforcer (i.e., those used in CET). For instance, a tied tourniquet and filled syringe (arranged by operant behavior), the odor of sulfur (which occurs by lighting a match), the sound of a butane torch lighter (established by finger and hand movements), and the visual presence of a candle flame and a spoon are all the result of a chain of operant responses emitted by the drug abuser, as discussed earlier. Such proximal cues involved in the drug-taking ritual evoke physiological changes but may not set the occasion for drug availability; however, they may function as operant conditioned reinforcers (i.e., stimuli not directly correlated with primary reinforcement but that maintain operant responding over extended time frames) for the responses that produce them, which in turn sets the occasion for the consequential response, which in turn is reinforced by yet another stimulus event, which then occasions the next response, and so on. Furthermore, drug abusers often invest significant time in seeking drugs, particularly illicit drugs—hence, drug-seeking behavior. In view of the time and response cost spent in procuring illicit drugs, much has been written concerning the behavioral economics of drugs as commodities (e.g., Bickel, Yi, Mueller, Jones, & Christensen, 2010). Drug-seeking behavior is often maintained on rather lean schedules of reinforcement. That is, the user does not always obtain the drug commodity on each drug-seeking attempt. Thus, when a drug supply is low, its demand increases, as does the cost (Hursh & Winger, 1995). For instance, Vukmir (2004, pp. 560–562) outlined several sophisticated drug-seeking repertoires that emerged (i.e., they were shaped) in opiate-dependent patients seeking pain management in general medicine (i.e., “doctor shopping”).

In summary, there are a plethora of exteroceptive cues that play functional roles in cue-reactivity and in the regulation of drug seeking and self-administration. These cues must be categorized according to how and where they functionally occur within the drug-seeking and drug-taking sequence as arrayed over time. The next section attempts to categorize these functions in addressing operant–respondent interaction.

Operant–Respondent Interaction

The examples in the previous section illustrate the complex interplay among operant S^D and Pavlovian CS functions that interact in regulating drug seeking and drug self-administration. A functional operant analysis of drug cue-reactivity and drug-seeking and drug-taking behavior must delineate stimulus antecedents connected to drug reward in terms of respondent and operant stimulus roles. As noted by Greeley and Ryan (1995), cue-reactivity analyses have perhaps overemphasized the respondent CS in terms of “incentive roles” and have fallen far short in addressing the complexity of operant stimulus control relationships. What follows next is a brief survey of some important distinctions among Pavlovian and operant stimuli and how they may interact in the regulation of drug seeking and drug self-administration.

The Operant S^D Is Functionally Independent From the Pavlovian CS

Operant S^Ds set the occasion for voluntary responses to be reinforced by their consequences. The “Open” sign and the presence of a bartender in the examples presented earlier both function in this manner. In a rodent paradigm, for instance, the presence of a light may signal that a lever press will be followed by food reinforcement. Thus, there are three terms: the S^D, the response (R), and the primary reinforcer (S^r+). S^Ds can harbor CS functions embedded in the three-term contingency (S^D: R → S^r+) as a result of the temporal establishment between the light (or tone) and the food pellet; that is, the light is paired with food via the operant response. A bottle or glass/mug sets the occasion for lifting and drinking (coupled with the smell or taste of ethanol) and precedes an ethanol effect. The sight of crack cocaine and a crack cocaine pipe also function as discriminative stimuli but may also function as a CS in eliciting respondents that evoke cue-reactivity. However, CS functions act independently from discriminative S^D functions and are unnecessary for operant discriminative control. A few examples follow.

Marcucella (1981) showed that Pavlovian-elicited key pecks in pigeons emerged subsequent to operant discriminative functions, and extinguished first. In that study, elicited key pecks were dissociated from operant key pecks despite comparable reinforcement magnitudes (i.e., between the CS → US and response → reinforcer) that varied only as a function of the operant response. Holman and Mackintosh (1981) used a Pavlovian blocking procedure to separate CS and S^D functions of stimuli embedded in the three-term operant contingency. In a Pavlovian blocking procedure, a CS₁ is established in the first phase of a conditioning experiment. During a second phase, CS₂ is added to CS₁ to form a stimulus compound, which is then paired with the US. Such an arrangement renders the CS₂ ineffective in promoting elicited responses (i.e., blocked). In the Holman and Mackintosh (1981) study, the stimulus that was first treated as a Pavlovian CS did not prevent a second stimulus, with which it was compounded in a second phase, from acquiring operant S^D functions; however, it “blocked” the CS function. This finding suggested that (1) the operant discriminative function of a stimulus can be dissociated from its CS function and, more important, (2) CS functions are not necessary for the development of operant stimulus control. Finally, in a series of experiments, Rescorla (1992, 1994, 1995), Rescorla & Colwill (1989), and Colwill & Rescorla (1990) have shown that the operant three-term contingency is not reducible to simple binary CS → CS relationships, but rather the S^D functions hierarchically in modulating the response–reinforcer contingency (R → S^r+). Thus, S^Ds cannot be established without the response–reinforcer relationship (R → S^r+) in place first. Furthermore, operant behavior is often maintained on various schedules of intermittent reinforcement (e.g., variable ratio or interval); therefore, S^Ds that occasion rather lean schedules of positive reinforcement do not function well as Pavlovian CSs (Marcucella, 1981). Yet, behavior under operant discriminative control is persistent and resistant to extinction (Nevin, 2009; Podlesnik & Shahan, 2010), that is, “compulsive” behavior. Although it is beyond the scope of the present paper to provide a more detailed review and analysis of the basic research on Pavlovian–operant interaction, it appears that Pavlovian operations are neither necessary nor sufficient for operant discriminative control in animal learning studies.

Returning to the example of the “Open” sign (or the presence of the crack house), such stimuli set the occasion for drug availability, and might even elicit physiological responses that culminate in “craving”; however, the underlying physiological changes that may or may not be elicited do not function as discriminative stimuli. On balance, as will be noted in the next section, Pavlovian processes can affect operant responsiveness.

Pavlovian–Instrumental Transfer

Although Pavlovian respondent functions may be neither necessary nor sufficient for the discriminative stimulus functions of the operant S^D, such Pavlovian functions do influence the rate of operant responding (Bower & Grusec, 1964); hence, Pavlovian–instrumental transfer (PIT). The conditioned emotional response paradigm (CER), first used by Estes and Skinner (1941), demonstrated that an aversive CS that was previously paired with a shock US suppressed the rate of food-reinforced operant responding in rats. By contrast, Mauro and Mace (1996) showed that a Pavlovian CS associated with the same appetitive-reinforcing outcome as an operant response increased response rate during operant extinction. Pavlovian stimuli can promote more resistance to extinction, that is, behavioral momentum (see Nevin & Grace, 2000; Podlesnik & Shahan, 2010). A study conducted by the current author (Troisi, 2006) used a Pavlovian drug discrimination procedure to demonstrate that interoceptive Pavlovian CSs modulate operant responding in rats. In that study, lever pressing was initially maintained on a variable interval (VI) schedule of food reinforcement. In the second phase, the levers were removed from the chambers and drug discrimination training took place. On some sessions, one drug (nicotine or EtOH, counterbalanced for stimulus role) signaled food delivery on a variable time (VT) schedule and methodologically functioned as a CS⁺, whereas on other sessions, the CS⁻ drug signaled no food delivery. When the levers were reinstalled, there was significantly greater lever pressing in the CS⁺ drug condition compared to the CS⁻ condition, but the effect was not substantial. In contrast, when the drugs were established as operant S^D and S^Δ, response rates were dramatically different and discriminative control among the drug conditions was substantial. Those data showed that, whereas interceptive Pavlovian CSs can modulate operant behavior that shares a common appetitive food reinforcer, interoceptive S^Ds have more dramatic impacts on the regulation of operant behavior.

With regard to drug abuse, Pavlovian CSs associated with drug reward may increase responsiveness (“incentive motivation”) to operant behavior under discriminative control of S^Ds that occasion drug seeking and drug self-administration. However, the vast majority of reports concerning conditioned reinforcement in drug self-administration studies with rats have conceptualized the conditioned reinforcer as a Pavlovian CS. Interestingly, a recent study with humans (Hogarth & Chase, 2012) found that Pavlovian-like cues that were initially paired with tobacco cigarettes subsequently had no influence on choice of tobacco and failed to impact rate of extinction of instrumental behavior previously maintained by tobacco cigarette reinforcement. Furthermore, smoking status had no interactive influence. It is plausible that if Hogarth and Chase had established an operant S^D for completing a schedule of reinforcement that predicted tobacco, there would have been substantial differences in choice and rates of extinction.

CS effects can be dissociated from instrumental stimulus control. For instance, in one PIT investigation, Corbit and Balleine (2003) demonstrated that a Pavlovian appetitive CS increased the rate of responding of the proximal, but not distal, response in a heterogeneous operant chain when rats were food restricted. Under satiety, the impact was reversed. This effect was specific to a CS that was paired with the food reinforcer and not a CS previously paired with sucrose. These results showed that Pavlovian stimuli have varied effects on different responses within an operant chain depending on where the response is in relation to the primary reinforcer and whether or not the reinforcer was “devalued” (rat sated). They concluded that Pavlovian and instrumental “incentive learning” act independently. It would be informative to carry out a similar study using a heterogeneous chain that is maintained by drug reinforcement. An investigation of this sort could be carried out when the rats are preloaded with the drug (devaluation) or not (for an appetitive food reward example, see Baker, Weisman, & Benniger, 2012). Would Pavlovian stimuli associated with one drug US impact distal and proximal instrumental responses maintained by the drug reinforcer differently? Would such differences be specific to one drug reinforcer but not transfer to a response sequence that results in a different drug reinforcer? The results of Corbit and Balleine's study suggest that Pavlovian stimuli likely have less influence on behaviors remote in the drug-seeking ritual than those that occur just prior to the drug effect. Discriminative stimuli are likely to have greater impacts on distal responses than are Pavlovian CSs. Interestingly, no study has yet to be conducted that has evaluated the impact of Pavlovian stimuli on behaviors embedded in extended operant heterogeneous chains. Such an evaluation will have striking translational importance for drug abuse.

Operant Conditioned Reinforcement

Much has been reported regarding the nature of operant conditioned reinforcement (see recent work by McDevitt & Williams, 2010; Shahan, 2010). What follows is not intended, in any way, to summarize the conditioned reinforcement literature. The reader should refer to a rather well written theoretical review of conditioned reinforcement by Shahan (2010) that discusses behavioral momentum and Fantino's delay reduction hypothesis of conditioned reinforcement. Suffice it to say, stimuli associated with primary reinforcers, including drugs, maintain operant responding and have been conceptually studied in the context of drug abuse (Alessi, Roll, Reilly, & Johanson, 2002; Schuster & Woods, 1968).

As noted in the previous section, much of the animal research on incentive learning regarding drug reinforcement postulates that conditioned reinforcers that maintain drug-seeking behavior are established by Pavlovian processes (Arroyo, Markou, Robbins, & Everitt, 1998; Di Ciano, Robbins, & Everitt, 2008; Morrison, Thornton, & Ranaldi, 2011). These sorts of investigations typically employ second-order schedules of reinforcement (and not heterogeneous operant chains) that result in drug delivery (for reviews, see Everitt & Robbins, 2000; Schindler, Panlilio, & Goldberg, 2002). When the drug is delivered, it is preceded by a brief exteroceptive CS. In all of these studies, conditioned reinforcement is conceptualized as a Pavlovian relationship. Unfortunately, this conceptualization has perpetuated the notion that Pavlovian relations are critical for relapse. Interestingly, as “incentive cues,” these CSs do not clearly reinstate extinguished drug-reinforced operant responding in rats when presented noncontingently on behavior (Di Cano & Everitt, 2003; McFarland & Ettenberg, 1997) but can reinstate extinguished responding if presented contingently on responding. Interestingly, an important study by Di Ciano and Everitt (2003) with rats that received heroin or cocaine contingent on an operant response revealed that an operant S^D reinstated extinguished responding either contingently on responding or noncontingently. These results show the importance of the response–reinforcer relationship as embedded in the three-term operant relationship, rather than the Pavlovian stimulus–reinforcer relationship, in playing a vital role in relapse. These findings are important for understanding how some aspects of conditioned reinforcement contribute to relapse. However, it is critical to note that although Pavlovian CSs can function as operant conditioned reinforcers, not all instances of operant conditioned reinforcement are established by Pavlovian relationships (e.g., those distal in a heterogeneous chain).

Operant S^Ds not only set the occasion for a response to be reinforced, they also reinforce novel behaviors for which they are a consequence—as conditioned reinforcers (D'amato, Lachman, & Kivy, 1958; Dinsmoor, 1985; Kelleher, 1966; Kelleher & Gollub, 1962; Lattal & Crawford-Godbey, 1985; Zimmerman, 1959). It is noteworthy that Shahan (2002) has reported some important empirical evaluations on the effectiveness of using novel responses to evaluate cues that are deemed conditioned reinforcers, but it is beyond the scope of the present paper to summarize those issues here. Nonetheless, perhaps the best example of how effective S^Ds are as conditioned reinforcers is revealed, again, by the heterogeneous operant chain in which there are multiple S^Ds that link multiple topographically different responses, as discussed earlier. Just as operant discriminative control is not predicated on Pavlovian operations, conditioned reinforcement is not reducible to Pavlovian CS → US relationships. As noted earlier, Pavlovian higher order stimulus control (Holland & Rescorla, 1975; Rizely & Rescola, 1972) is not evident beyond second-order conditioning (Gewirtz & Davis, 2000), whereas operant chains have numerous links coupled by multiple discriminative stimuli (Lattal & Crawford-Godbey, 1985). Thus, distal S^Ds in long heterogeneous operant chains are not likely to evoke respondents; however, they can set the occasion for the more proximal response and also maintain the response for which they are consequential. In fact, Pavlovian respondent CS → US relationships can even be dissociated from operant conditioned reinforcement. For instance, Parkinson, Roberts, Everitt, and Di Ciano (2005) carried out an operant investigation in which rats' lever pressing produced a CS that was paired with an appetitive operant reinforcer. During the second phase, the food reinforcer was devalued with lithium chloride (taste aversion). Despite devaluation of the reinforcer, the CS functioned as an effective conditioned reinforcer for a novel response. Thus, there appears to be clear dissociation between the CS → US relationship and the response–reinforcing relationship established between the operant response and the CS-treated operant conditioned reinforcer. Exposure to S^Ds following extinction of drug-reinforced responding evokes strong recovery (McFarland & Ettenberg, 1997); however, to date, not one investigation has evaluated the impact that an S^D (as conditioned reinforcer) has on reinstatement of extinguished drug-reinforced operant responding embedded in an operant chain. Such an evaluation may be important in furthering the understanding of relapse.

The Observing Response

Earlier it was noted that drug abusers often spend significant amounts of time searching for illicit drug commodities. Certainly, drug abusers must seek out and respond differentially to stimuli that function as discriminative stimuli and occasion drug availability (i.e., “foraging”). The “Open” sign in the example presented earlier always occasions alcohol availability, but the presence of drug abusing peers or the presence of a crack house or a shooting gallery may only intermittently occasion the availability of illicit commodities, as also addressed earlier. The observing response may concern drug-seeking behavior.

In rats, Dinsmoor (1983, 1985) demonstrated that lever pressing on one manipulandum (the observing response; OR) was maintained by an exteroceptive stimulus that was not directly associated with the primary reinforcer but rather “informed” the organism that reinforcement was contingent on responding on a second manipulandum, one that directly led to the primary reinforcer (see also Wyckoff, 1952). ORs do not alter the probability of reinforcement that maintains the primary response. ORs are thus maintained by the conditioned reinforcing functions of the operant S^D (Case & Fantino, 1981) and indicate whether or not the S^D component is in effect (a “look and see if reinforcement is available” response). The individual is likely to only pull up to a bar late at night if the “Open” sign is lighted, and if it is not, he or she may continue seeking out other lighted “Open” signs that indicate alcohol availability. A crack cocaine user may telephone a drug dealer (the observing response) but not seek out the dealer if the phone is not answered. Similarly, a heroin abuser may respond differentially to social stimuli that function as conditioned reinforcers.

ORs have been shown with drug reinforcement in rats, with ethanol (Shahan, 2002, 2003), and in rhesus monkeys, with cocaine and opiates (Woods & Winger, 2002). In both studies, a self-administration (SA) response produced the drug under limited time constraints (the S^D component). When the S^D component was not in effect, the SA response was not reinforced. The OR was reinforced by an exteroceptive stimulus that was not otherwise present only when the S^D component was in effect, in which case, the SA response was drug reinforced on a particular schedule of reinforcement. An observing response analysis may parallel that which occurs with regard to drug-seeking behavior compared to drug-taking behavior. Although subtle, this point has also been implied by Shahan (2002). Several important theoretical questions follow: In humans, do conditioned reinforcers that maintain observing responses evoke a different form of drug cue-reactivity than basic CS-paired cues associated with drugs? Do Pavlovian stimuli that evoke drug cue-reactivity increase observing behavior that is maintained by conditioned reinforcers that occasion drug self-administration? Does re-exposure to observing stimuli evoke drug-seeking behavior following extinction (see Thrailkill & Shahan, 2012)? These sorts of questions and the results from empirically evaluated investigations that entertain them might translate into more effective treatment strategies. Drug addicts certainly respond differentially to remote discriminative stimuli that occasion long heterogeneous operant chains that eventuate in drug reinforcement. Reducing these phenomena to respondent operations (although such operations influence operant responsiveness to such stimuli; see the next section) will do little to lead to effective drug-treatment strategies.

Establishing/Motivating Operations (EOs)

Many interoceptive events have been shown do evoke cue-reactivity and “craving” and temporarily increase the reinforcing value of a drug of abuse. As noted earlier, various emotional states often evoke cue-reactivity (Tiffany, 2009). Thirst, hunger, fatigue, and stress have all been shown to increase (reinstate) extinguished responding that was previously maintained by drug reinforcement. The most obvious example of this is drug withdrawal. In a hallmark article, Michael (1982) explicated the distinction between stimuli that function as operant S^Ds and those that establish more general motivating operations. For example, Michael argued (and see also McDevitt & Fantino, 1993) that electric shock does not function as an operant discriminative stimulus for rats' lever pressing but rather (as a stimulus) establishes the operation by which shock removal functions as an operant negative reinforcer. Similarly, food deprivation, which also produces a set of interoceptive states (e.g., Davidson, 1998) is not a discriminative stimulus for lever pressing but does establish the operation by which a food pellet functions as a positive reinforcer. In view of these distinctions, various emotional states or drug withdrawal states do not function as operant S^Ds for drug-reinforced responding but may establish the operation by which reductions in such states function as negative reinforcers (e.g., relief drinking). Fatigue might establish the operation by which a psychostimulant functions as a negative reinforcer. For instance, Silverman, Kirby, and Griffiths (1994) demonstrated that the reinforcing effects of sedatives and stimulants were modulated by the experimental work requirements (relaxation task vs. vigilance task) in human participants with histories of sedative and stimulant abuse. The work requirements thus established the operations by which the drug states functioned as reinforcers and changed over time.

Returning now to the example involving the “Open” sign outside of a bar: Thirst, anger, or some other emotional reaction (or a combination) may function as an EO that temporarily alter the evocative ability of the “Open” sign to set the occasion for the operant chain that follows. Although these EOs are not Pavlovian CSs, such EOs may be CRs that are elicited by other antecedent stimuli. If the individual was not thirsty and angry, responsiveness to the “Open” sign would be expected to be much lower. Of course, the physiology of thirst and emotion is complex and mediated by many variables. By analogy, methadone maintenance is designed to prevent opiate withdrawal, which in turn decreases responsiveness to cues that signal opiate availability (Langleben et al., 2008). Methadone maintenance is an EO (or perhaps an abolishing operation) that temporarily lowers the reinforcing valence of illicit opiates. Most forms of drug replacement therapy may function as abolishing operations, the removal from which may function as establishing operations that not only increase the reinforcing value of the drug of choice (see Laraway, Snycerski, Michael, & Poling, 2003, pp. 408–409) but also may increase responsiveness to drug-related exteroceptive and interoceptive cues (Langleben et al., 2008).

Responding in the presence of the discriminative stimulus does not extinguish if the reinforcer has been temporarily devalued (e.g., operant extinction under satiety). Recently, Troisi, Bryant, and Kane (2012) showed that extinction of responding under the discriminative stimulus effects of nicotine was more effective when rats were maintained on a restricted feeding schedule rather than when sated. Rats that were placed back on restricted feeding schedule, following extinction while stated, showed more recovery of discriminative control compared to when they were tested while stated. Drug replacement therapy coupled with CET exposure would be fruitless from the perspective of an establishing operations analysis. Without classifying and taking into consideration the impact of other EOs (cues that promote cue-reactivity) on drug-taking behavior, designing extinction trials would be somewhat limited in effectiveness.

As noted earlier, Pavlovian contingencies can modulate ongoing operant activity (e.g., PIT). Corbit and Balleine (2003) showed that CSs impacted proximal but not distal responses in a heterogeneous operant chain depending on whether or not the reinforcer was devalued (i.e., abolishing operation). Pavlovian drug-opposite CRs and conditioned withdrawal reactions may be conceptualized as conditioned EOs (cf. Michael, 1993, for slightly different perspective). Corbit and Balleine's demonstration suggests that CSs connected with the drug effect may have less impact on responses and conditioned reinforcers most distal in the heterogeneous drug-taking sequence and perhaps on observing behavior than on the actual self-administration response. More empirical demonstrations of these ideas are needed. A functional EO analysis of cue-reactivity and “craving” may lead to better treatment strategies in the future (see Troisi et al., 2012).

As a final note on establishing/motivating operations, the behavior analytic community appears to have done a remarkable job of addressing philosophical underpinnings of establishing operations, but unfortunately little has become of it—other than in the education literature published in the Journal of Applied Behavior Analysis (see Lotfizadeh, Edwards, Redner, & Poling, 2012) Here, I offer what may be a sound applicable outlet for the establishing operations construct—drug abuse. I have cited a more recent review of the establishing operations issue (Laraway et al., 2003). In that review, in one small section, only two studies were conceptualized in the EO framework: drug replacement therapy as an abolishing operation. To this author's knowledge, neither of those empirical behavioral pharmacology studies used Jack Michael's EO construct. It seems that there are more theoretical papers pertaining to the EO construct (in particular whether the mechanic and the screw driver request to the assistant is really an apt metaphor for the construct) than actual empirical demonstrations in the animal literature. That said, the Pavlovian literature regarding reinforcer devaluation has done a spectacular job on this issue, although it is not labeled as such. The axioms may differ, but the underlying functions are identical—and important in the context of drug abuse (see Troisi et al., 2012). Cue-reactivity is an establishing operation, and multiple stimulus effects, both interoceptive and exteroceptive in nature, interact in this operation. Pavlovian stimulus control is but one means to promote cue-reactivity and craving.

Operant Approaches to Drug Abuse Treatment: Some Caveats

Conceptualizing drug seeking and drug taking as an operant chain and carrying out extinction of such a chain may be a better approach to drug abuse treatment than Pavlovian-based CET. The preceding sections were intended to illustrate the dynamic interaction among respondent and operant learning functions that likely modulate drug abuse. However, operant responding is also subject to spontaneous recovery, context renewal, and reinstatement (Nakajima & Tanaka, 2000; Rescorla, 1997; Troisi, 2003a, 2003b). This can pose a rather pessimistic picture for relapse prevention (see also Bouton, 2002). It could be that the best hope for behavioral treatment approaches to drug abuse is not only in terms of extinction of the operant three-term contingency in the presence of operant drug-related EOs, S^Ds, and CSs that evoke craving and set the occasion for drug-reinforced responding but also by mitigating those relationships with response alternatives that are maintained by qualitatively different reinforcers.

Omission Training With Vouchers for Drug Abstinence

Omission training usually involves the reinforcement of the non-occurrence of a target behavior. In the laboratory, for example, a differential reinforcement of other behavior (DRO) schedule can be put in effect. The reinforcer is contingent on the non-occurrence of a lever press within a specified time interval. However, if the response occurs within the specified interval, the timing interval is reset. Methadone maintenance programs have at least partially functionally employed this basic operation (Farrell et al., 1994). Clients are required to provide urine samples. If the sample test is negative, the prescribed dose of methadone is dispensed. The contingencies that are in place for positive samples vary from treatment unit to treatment unit but can involve forfeiture of the dose on a given day.

In the 1990s, attempts were made to provide greater incentive for non-drug use in cocaine- and opiate-abusing populations (i.e., drug-free urine samples). Programs at the University of Vermont and Johns Hopkins began implementing a token economy–like approach to drug abuse treatment. Again, the attempt here is not to provide a detailed review of this rather extensive literature. In its basic form (e.g., Higgins & Silverman, 2008; Silverman, Robles, Mudric, Bigelow, & Stitzer, 2004), drug abusers receive vouchers for non-drug use. The vouchers can be exchanged for goods in the community (other than drug commodities). The cash value of these vouchers is increased with consecutive drug-free urine samples and is decreased with positive urine samples (e.g., Rogers et al., 2008). These programs have had considerable success as judged by the duration of abstinence (drug-free urines) during treatment (Stitzer & Petry, 2006) and according to two meta-analyses (Lussier, Heil, Mongeon, Badger, & Higgins, 2006; Prendergast, Podus, Finney, Greenwell, & Roll, 2006). These two investigations also summarized the usefulness of voucher programs for treating several drugs of abuse (opiates, cocaine, alcohol, tobacco).

Response Alternatives

In addition to reinforcing drug abstinence, the effectiveness of reinforcing alternative non-drug–related behaviors has also been evaluated. For example, skill training has been implemented (Silverman, 2004; Silverman, Roll, & Higgins, 2008). Such strategies capitalize on shaping alternative non-drug–related behaviors with alternative (non-drug) monetary reinforcers. One specific approach, the therapeutic work environment (DeFulio et al., 2012; DeFulio & Silverman, 2011), trains poor and chronically unemployed individuals with histories of opiate and cocaine abuse to engage in general work-related skills and professional skill development (e.g., computer data entry skills), which are reinforced. One investigation by Silverman, Svikis, Wong, Hampton, Stitzer, and Bigelow (2002) found that 54–60% of the sample populations continued to provide “clean” samples 3 years following completion of the study. Other general “cognitively” based coping skills training (CST) have also been effective for the treatment of drug abuse (e.g., O'Leary & Monti, 2002).

Taken together, it seems that reinforcing drug abstinence and/or shaping alternative behaviors with non-drug reinforcers (albeit monetary in nature) is promising for drug abuse treatment. It certainly eclipses the utility of CET in its basic form. Assuming that the reinforcement contingencies for these alternative behaviors remain in place so as to not undergo extinction, this treatment endeavor appears promising, as long as funding for them is sustained. However, extinction of alternative behaviors may lead to relapse.

The Caveats: Behavioral Momentum, Resurgence, and the Return of Cue-Reactivity

As a preface, Tim Shahan and colleagues have done a remarkable job capturing behavioral analytic evaluations of relapse. Some of the major themes are briefly outlined here (see also Bouton et al, 2012; Winterbauer, Lucke, & Bouton, 2013). The clinical research communities should track the relevance of these reports to improve clinical efficacy.

Behavioral momentum

Behavior resists change. Nevin (1988, 1993, 2009) proposed a rather elegant metaphor based on a law from classic physics—momentum. Using different reinforcement schedules, Nevin quantified the degree to which the rate of operant responding resists change during extinction, or during change of reinforcement schedule. Generally speaking, the leaner the reinforcement schedule, the less the behavioral momentum (for a comprehensive review, see Nevin & Shahan 2011; Shahan, 2010). However, somewhat counterintuitively, among drug abusers with long-term histories of drug reinforcement, drug-seeking → drug-taking rituals may certainly acquire significant “behavioral momentum” over illicit drugs, which, as discussed previously, are likely to be maintained on rather lean schedules of intermittent reinforcement and resist changes in reinforcement contingencies (see Podlesnik & Shahan, 2010). Thus, extinction of complex drug-seeking and -taking rituals should require extensive non-reinforcement training to sustain zero, or near-zero, rates of responding to minimize the probability of relapse (cf., Quick & Shahan, 2009). Alternative behaviors used in operant drug treatment would also need to acquire significant momentum to resist change (Pyszczynski & Shahan, 2011). It could be that changing the reinforcement contingencies in the therapeutic work environment and voucher programs from fixed to variable might promote greater resistance to extinction (see Winterbauer et al., 2013) if the reinforcement contingencies abruptly changed (funding diminishes or is not increased). Furthermore, Pavlovian cues for differing non-drug reinforcers in differing contexts could also influence the extent to which the novel behaviors associated with non-drug reinforcement persist or extinguish and older drug-related behaviors reemerge (see the next section; Pyszczynski & Shahan, 2011).

Behavioral resurgence

Behavioral resurgence, first observed by Epstein and Skinner (1980), is the return of older extinguished behavior when recently acquired novel behavior undergoes extinction (for a recent study, see Shahan & Sweeney, 2011). Countering older drug-taking behaviors with novel alternative behaviors (i.e., skill training and therapeutic work environment) that are maintained by qualitatively different non-drug–reinforcing outcomes appears to be more effective than CET. However, novel behaviors, or previously established non-drug–related “pro-social” behaviors, are often acquired or recover slowly as sensitization to novel non-drug rewards ensues (Robinson & Berridge, 1993, 2003). Such skills would need to acquire substantial behavioral momentum (see suggestions by Nevin, 1993) to resist potential (sometimes abrupt) changes in reinforcement contingencies, the absence of which (i.e., extinction) would likely promote behavioral resurgence (Bouton et al, 2012; Epstein & Skinner, 1980; Podlesnik, Jimenez-Gomez, & Shahan, 2006; Podlesnik & Shahan, 2010; Rescorla, 1995; Winterbauer & Bouton, 2010; Winterbauer et al, 2013) of the older drug-seeking and drug-taking behaviors (i.e., relapse).

Behavioral resurgence has only recently been applied to drug abuse (Bouton et al, 2012; Podlesnik et al., 2006; Podlesnik & Shahan, 2010; Quick, Pyszczynski, Colston, Shahan, 2011; Winterbauer & Bouton, 2010; Winterbauer et al, 2013). For example, Podlesnik et al. (2006) conducted a study using ethanol-maintained responding (lever pressing) in rats. In a second phase, an alternative response (chain pull) was established and was maintained by food reinforcement. In the third phase, the alternative food-reinforced response was extinguished. As predicted, lever pressing resurged, but there was greater resurgence of behavior previously maintained on rich schedules of reinforcement where reinforcement occurred often. More studies of this type using IV drug infusion and brain-stimulation reward are warranted.

A return of cue-reactivityas an EO

Changing the reinforcement contingencies may modify drug-seeking and drug-taking behavior as described previously. However, it is plausible that abrupt changes in reinforcement contingencies (extinction) of newer, non-drug behaviors might increase responsiveness (cue-reactivity) to drug-related exteroceptive (and interoceptive) stimuli, including discriminative stimuli and Pavlovian CSs, described in detail throughout this paper. Under such circumstances, the reinforcing value of a drug would likely be increased; in other words, such conditions might temporarily establish the operation by which a drug, again, becomes a reinforcer (i.e., establishing operations). Would an individual be more vulnerable to relapse under these conditions?

A Hybrid Proposition

Operant-based treatment approaches have focused on (a) reinforcing the non-occurrence of drug-taking behavior (methadone maintenance, contingency contracting) and (b) establishing alternative non-drug–related behaviors (therapeutic work environment, skill training). However, these treatments have made no attempt at exposing drug abusers to drug-related cues that provoke cue-reactivity (i.e., EO induction;S. Higgins, personal communication, February 5, 2011) This final section proposes a hybridization of these two models.

The Proposal

An integrated behavioral approach might (following detoxification of the client) first carry out extinction of the drug-seeking ritual by allowing the drug user to seek out exteroceptive cues correlated with drug commodities. This could be accomplished using a virtual environment with audiovisual displays (Culbertson, 2012; Rothbaum, 2005) of the clients' natural drug-seeking and drug-taking environments (e.g., bar or crack house). Observing responses could be integrated into these virtual environments and then extinguished. Standard measures of craving would be recorded. During Phase II, drug self-administration rituals could be extinguished in the standard CET arrangement, except that the self-administration response would need to be followed by a placebo. This phase would need to be carried out extensively with many, many, extinction sessions over an extended time frame, and an extinction criterion would need to be achieved before entering the third phase. Phase III would utilize a voucher-based therapeutic work environment, skill training, or the general shaping of alternative behaviors. Depending on social demographics, other sorts of response alternatives that would vary from individual to individual could be established under specific EOs evoked by drug cues (e.g., music, gaming, exercise, sporting). For instance, in a relatively recent study, wheel running was shown to reduce reinstatement of cocaine seeking in rats (Zlebnik, Anker, Gliddon, & Carroll, 2010). In the final phase of this hybrid therapy, attempts to evoke cue-reactivity would be made by re-exposing the individual to drug-seeking– and drug-taking–related stimuli in the laboratory (including the VR) and in the community. In this phase, the response alternatives would be executed under conditions that promote cue-reactivity (as measured electrophysiologically and by self-reported craving). (Note: Similar studies have used CET with skills training [Monti & Rohsenhow, 1999; Monti et al, 1993; Rohsenhow et al., 2001], but not in the framework proposed here.) This final phase would also need to be carried out over the long term and would likely require the client to return to the conditioning lab for many visits (“booster visits”; see Bouton, 2002) over many years, with repeated exposure to drug stimuli and the execution of motivating operations that, under verbal stimulus control, set the occasion for the emission of non-drug–related behavioral response alternatives.

What Can Behavior Analysis Learn From 12-Step Programs?

Interestingly, 12-step programs appear to, at least functionally, advocate the approach just described, albeit more simply and in the vernacular. For example, Kelly, Stout, Magill, and Tonigan (2010; and see Kelly & Yeterian, 2012) showed the effectiveness of social reinforcement contingencies in Alcoholics Anonymous. Those in recovery are encouraged to learn various catchphrases such as, “keep it simple,” “the first drink gets you drunk,” “keep coming,” “first things first,” “meeting makers make it,” “one day at a time,” “ask for help,” “easy does it,” and many others. These “slogans” represent verbal scripts that are reinforced verbally/socially within the social therapeutic community and can then be easily executed outside of the community in previous drug-taking exteroceptive contexts or under emotional states previously related to drug/alcohol use (see McCrady, 1994). Functionally speaking, these represent novel repertoires; they are response alternatives that are shaped naturally within the therapeutic community by alternative social reinforcers. Tangible incentives (medallions and chips) for various lengths of sobriety are also often used as well.

As is the case among exteroceptive stimuli, multiple interoceptive events interact, as described above, and form a unique cue—a sensory gestalt. In 12-step programs, recovering drug abusers and/or alcoholics are advised to beware of HALTT, an acronym for hungry, angry, lonely, thirsty, and tired, all common interoceptive stimulus conditions reported to evoke drug craving and prompt relapse under particular circumstances (Alcoholics Anonymous), which in the past may have been present during drug self-administration or alcohol consumption (cf. McCrady, 1994; McGee, 2000; O'Connor & Schottenfeld, 1998). For example, a recent study (Puhl, Fang, & Grigson, 2009)reported that acute sleep deprivation reinstated cocaine self-administration in rats. Negative affect has been shown to impact CET (e.g., Stasiewicz et al., 1997), and interactions between mood and exteroceptive social cues have been entertained (Laberg, 1990). This poses a rather formidable and perhaps bleak prognosis for anyone in recovery, as any number of combinations of interoceptive stimuli can interact with proximal and/or distal exteroceptive stimuli that once occasioned drug taking or alcohol consumption. It's a “recipe for relapse.” Certainly, the profiles of such interactions must vary pharmacologically from drug class to drug class, across methods of abuse and specific withdrawal states associated with specific drugs of abuse. Are these interoceptive responses EOs? During periods of relapse vulnerability, members are encouraged to call sponsors or other members within the community, or simply attend a meeting. Because such meetings have no financial cost and are available in most cities throughout the world, the execution of these repertoires is readily occasioned within the social therapeutic context (McKellar, Stewart, & Humphreys, 2003). Relapse vulnerability, promoted by a sensory gestalt (HALTT), might set the occasion for alternative behavior that minimizes the probability of drug taking and, hence, relapse. This final section, of course, was not intended to suggest that 12-step programs should be alternatives to a sound functional analysis of drug abuse but rather to note the extent to which a functional behavioral approach may need to go to attain better clinical efficacy for the treatment of drug abuse.

Conclusion

Historically, Pavlovian extinction (CET) has been one treatment strategy for drug addiction. Unfortunately, the best available data, to date, have demonstrated CET's lack of clinical efficacy. Drug seeking and drug taking may be better conceptualized as complex heterogeneous operant chains in which voluntary behavior is linked by operant discriminative stimuli (not Pavlovian CSs) that function as operant conditioned reinforcers. Pavlovian operations modulate responsiveness to operant stimuli but may not mediate drug-seeking and drug-taking behavior. Extinction of the drug-seeking and drug-taking behavior within the operant chain in the presence of respective discriminative stimuli and conditioned reinforcer (under different establishing operations, i.e., cue-reactivity) coupled with the shaping of response alternatives (voucher programs, therapeutic work environment, skill training) is a proposed alternative treatment strategy. CET should incorporate a more operant-based operating system, and operant drug treatment programs should utilize cue-exposure/reactivity to more effectively treat drug addiction.