BEHAVIORAL MECHANISMS UNDERLYING NICOTINE REINFORCEMENT

Abstract

Cigarette smoking is the leading cause of preventable deaths worldwide and nicotine, the primary psychoactive constituent in tobacco, drives sustained use. The behavioral actions of nicotine are complex and extend well beyond the actions of the drug as a primary reinforcer. Stimuli that are consistently paired with nicotine can, through associative learning, take on reinforcing properties as conditioned stimuli. These conditioned stimuli can then impact the rate and probability of behavior and even function as conditioning reinforcers that maintain behavior in the absence of nicotine. Nicotine can also act as a conditioned stimulus, predicting the delivery of other reinforcers, which may allow nicotine to acquire value as a conditioned reinforcer. These associative effects, establishing non-nicotine stimuli as conditioned stimuli with discriminative stimulus and conditioned reinforcing properties as well as establishing nicotine as a conditioned stimulus, are predicted by basic conditioning principles. However, nicotine can also act non-associatively. Nicotine directly enhances the reinforcing efficacy of other reinforcing stimuli in the environment, an effect that does not require a temporal or predictive relationship between nicotine and either the stimulus or the behavior. Hence, the reinforcing actions of nicotine stem both from the primary reinforcing actions of the drug (and the subsequent associative learning effects) as well as the reinforcement enhancement action of nicotine which is non-associative in nature. Gaining a better understanding of how nicotine impacts behavior will allow for maximally effective tobacco control efforts aimed at reducing the harm associated with tobacco use by reducing and/or treating its addictiveness.

2.0 Introduction

Cigarette smoking is the leading cause of preventable morbidity and mortality worldwide, resulting in about 4.9 million deaths per year. It is widely accepted that nicotine is the primary psychoactive and reinforcing component of tobacco that produces the addictive state underlying sustained use of cigarettes and other tobacco products (Stolerman & Jarvis, 1995; USDHHS, 1988).

Despite the clinical observations that tobacco products are quite addictive and success rates for quitting tobacco use are low (USDHHS, 1988, 2012), experimental evidence suggests the primary reinforcing properties of nicotine, by itself, are weak. Indeed, when considering the primary reinforcing properties of nicotine compared to other drugs of abuse, the addictive power of tobacco products is surprising. However, the reinforcing properties of nicotine are much more complex than simple primary reinforcement from the drug. In addition to nicotine acting as a primary reinforcer, other stimuli that are consistently paired with nicotine can, through associative learning, take on reinforcing properties as conditioned stimuli. These conditioned stimuli can then impact the rate and probability of behavior and even function as conditioning reinforcers that maintain behavior in the absence of nicotine. Nicotine can also act as a conditioned stimulus, predicting the delivery of other reinforcers, which may allow nicotine to acquire value as a conditioned reinforcer. These associative effects - establishing non-nicotine stimuli as conditioned stimuli with discriminative stimulus and conditioned reinforcing properties as well as establishing nicotine as a conditioned stimulus, are predicted by basic conditioning principles. However, nicotine can also act non-associatively. Nicotine directly enhances the reinforcing efficacy of other reinforcing stimuli in the environment, an effect that does not require a temporal or predictive relationship between nicotine and either the stimulus or the behavior. Hence, the reinforcing actions of nicotine stem both from the primary reinforcing actions of the drug (and the subsequent associative learning effects) as well as the reinforcement enhancement action of nicotine which is non-associative in nature. Together, these two actions constitute what has been referred to as the “dual-reinforcement” model of nicotine reinforcement (Caggiula et al., 2009; Chaudhri et al., 2006b; Donny et al., 2003). Nicotine as a conditioned stimulus/reinforcer has received less attention, but potentially functions as a third mechanism underlying nicotine reinforcement (Bevins, 2009). These actions serve as the basis for this chapter.

3.0 Nicotine self-administration

The gold standard for studying the reinforcing properties of drugs in experimental animals is self-administration in which animals need to perform a behavioral task (e.g., press a lever, poke their nose into a hole) to obtain the drug. Hence, the focus of the chapter is on self-administration studies. Data from other procedures such as conditioned place preference (CPP), in which animals choose between an environment previously paired with the drug and one never paired with the drug, are incorporated when they provide additional insight. Most nicotine self-administration studies have used rats (Corrigall & Coen, 1989; Donny et al., 1995), although self-administration has been demonstrated across a wide range of species including humans, non-human primates, dogs, and more recently, mice (Corrigall, 1999; Fowler & Kenny, 2011; Goldberg et al, 1983; Le Foll & Goldberg, 2005; Matta et al., 2007; Stolerman, 1999).

However, it is important to appreciate that depending on the details of the methodological approach, multiple actions of nicotine might be occurring together and underlie the observed behavior. Historically, nicotine self-administration procedures in rats have utilized a protocol in which nicotine delivery is paired with a stimulus such as a cue light, chamber light, or tone. That stimulus might function as a primary reinforcer and/or acquire conditioned reinforcing properties after repeated association with nicotine infusion delivery. The reinforcing effects of that stimulus might then also be affected by the non-associative effects of nicotine. Thus, self-administration procedures often do not distinguish between primary reinforcement, reinforcement enhancement, and associative effects that may emerge over repeated experimental sessions. When considering data from nicotine self-administration studies, it is essential to understand the details of the methodological approach and be mindful of the different actions of nicotine that may underlie the observed behavior. This complexity is arguably an appropriate model for tobacco use in people in which the interaction between nicotine and concurrent stimuli is reality and these multiple reinforcing actions of nicotine are integrated to support behavior.

The reinforcing effects of nicotine are affected by a number of important moderating variables. For example, the duration and route of administration of nicotine delivery and the sex and age of subjects impacts the reinforcing properties of nicotine, and are highly relevant when considering the reinforcing properties of tobacco products. Furthermore, in the context of tobacco products, other chemical constituents found in tobacco products that would be delivered along with nicotine may interact with nicotine to alter its complex reinforcing properties. Moderating variables are discussed in more detail below.

Although humans typically self-administer nicotine in the context of tobacco products, laboratory studies have shown than people will work for nicotine. This research, typically conducted with infusions of nicotine paired with a novel light stimulus, shows that nicotine supports behavior that is dose- and schedule-dependent and different from vehicle (Harvey et al., 2004; Henningfield & Goldberg, 1983). Smokers also report subjective experiences consistent with the rewarding effects of nicotine (Sofuoglu et al., 2008). Although limited, these human laboratory studies confirm the basic observations from experimental animals, supporting the validity of the preclinical models that form the basis of this chapter.

3.1 Outcome measures

It is worth noting that while most self-administration studies focus on measuring response or infusion rates during stable periods of self-administration as the key dependent measures, there are several other important measures that may reflect the reinforcing properties of nicotine. In particular, the rate at which rats acquire self-administration likely reflects the reinforcing actions of nicotine (i.e., the more reinforcing, the faster rats will acquire the behavior). Similarly, the percentage of rats self-administering nicotine should also reflect the reinforcing properties of nicotine under those conditions. Like stable rates of nicotine self-administration that are maintained over multiple sessions, the rate and percentage of rats acquiring is dose-related and might be particularly sensitive to changes in the reinforcing properties of nicotine. For example, self-administration on a fixed ratio (FR) 2 schedule for a low dose of nicotine (7.5 μg/kg/infusion) compared with a maximally-effective dose (60 μg/kg/infusion) develops more slowly and in a smaller percentage of rats, but the level of self-administration ultimately attained is similar (Smith et al., 2014b). Many researchers also measure the willingness of an animal to work for a reinforcer, in this case infusions of nicotine, by increasing the number of responses required for an infusion within a session (progressive ratio [PR]) (Donny et al., 1999). Other measures such as the latency to start responding or earn an infusion, responding during a time out period, or responding despite an additional negative consequence have received relatively little attention but might provide additional insight into the factors controlling behavior (e.g., latency to first infusion may be a reflection of the influence of the context on behavior).

3.2 Route, dose, and rate of nicotine

The pharmacological actions of nicotine are mediated through its interactions with nicotinic acetylcholine receptors (nAChR), which are located in many sites throughout the brain and rest of the body (Leslie et al., 2013). Multiple subtypes of nAChR comprised of many subunit compositions have differing affinities for nicotine and kinetics (Picciotto et al., 2001). Thus, it is not surprising that the reinforcing actions of nicotine are dependent upon dose, route of administration, and temporal aspects of delivery (Le Foll & Goldberg, 2005).

Nicotine self-administration protocols predominantly use methods that deliver infusions intravenously over a brief period of 1-5 sec (Matta et al., 2007). The use of short duration intravenous (iv) infusions is based on the understanding that drugs that rapidly reach the brain are considered more reinforcing and have increased abuse liability (Benowitz, 1990; Samaha & Robinson, 2005), and the assumption that nicotine inhaled from cigarette smoke is rapidly absorbed into the pulmonary circulation and thus reaches the brain 5-10 sec following inhalation (Russell & Feyerabend, 1978). Additional puffs of cigarette smoke provide additional pulses of nicotine, superimposed on an increasing blood level of nicotine (Rose et al., 1999). Importantly, rapid discrete iv infusions of nicotine support self-administration (Corrigall & Coen, 1989; Donny et al., 1995; Matta et al., 2007). In self-administration procedures that use cues (e.g., a cue light located above the active lever or other operandum associated with iv nicotine delivery), rats reliably self-administer nicotine in the 10-60 μg/kg/infusion range (Chaudhri et al., 2007; Donny et al., 1995, 1998, 1999). Several studies directly examining the impact of prolonging infusion duration on self-administration of nicotine (Wakasa et al, 1995; Wing & Shoaib, 2013) determined that short duration infusion delivery supported greater responding, as might be expected from other abuse liability research (Ator & Griffiths, 2003; Busto & Sellers, 1986; Farre & Cami, 1991; McColl & Sellers, 2006; Samaha & Robinson, 2005).

On the other hand, it has recently been argued that the rapid iv infusion of nicotine (<5 sec) typically used in rat self-administration studies, coupled with single infusion doses that are on par with the quantity of nicotine delivered in an entire cigarette (i.e., 10-30 μg/kg/cigarette for a 70 kg individual), does not accurately represent the increase in plasma nicotine concentration of a person smoking a cigarette (Sorge and Clarke, 2009a; Rose et al., 1999). To model the dose of a single cigarette puff and the seconds-to-minutes long rise in arterial nicotine concentration, smaller doses delivered over a longer infusion duration may be required. “Puff-sized” doses modeling that delivered in a single puff of a cigarette might be closer to 1-10 μg/kg nicotine and the iv infusion duration might be several tens of seconds to better model tissue nicotine delivery from smoking cigarettes. Sorge and Clarke (2009) performed a series of experiments designed to more closely mimic nicotine delivery in human smokers and determine optimal dose/duration combinations chosen by rats in a concurrent access protocol. First, to test the reinforcing value of nicotine (15 μg/kg/infusion) delivered over different lengths of time, adult male rats self-administered infusions delivered over 3, 30, 60, or 120 sec in a cued protocol. On a FR 1 schedule of reinforcement, there was no effect of infusion duration on infusions earned; however, on an FR 5 schedule, rats self-administered equal numbers of infusions of 3 and 30 sec lengths, which were significantly greater than more prolonged infusion durations. Rats given the opportunity to respond on different levers to deliver 3 sec or 30 sec infusions of nicotine (15 μg/kg) selected the 30 sec infusion duration significantly more than the 3 sec duration (10 versus 6 infusions during 2 hr sessions). Analysis of the dose-response effects of nicotine using this prolonged 30 sec infusion duration determined that on an FR 1 schedule, doses between 3-30 μg/kg supported self-administration, and all doses above 3 μg/kg supported self-administration on an FR 5 schedule of reinforcement. The authors concluded that rats prefer slow infusions to fast when given a choice and that rats will self-administer a wider range of nicotine doses, including “puff-sized” doses, when the infusion duration is prolonged to this 30 sec range.

Other data, however, are inconsistent with this observation. The low doses shown to support behavior with prolonged infusions (i.e., 3 and 10 μg/kg/infusion), have also supported robust self-administration with 1 sec infusions in cued paradigms (Bardo et al., 1999; Donny et al., 1995; Smith et al., 2013, Smith et al., 2014b). Furthermore, unpublished data from our laboratory comparing infusions of nicotine delivered over 1 sec or 20 sec found little difference in responding for both 3.75 and 10 μg/kg/infusion nicotine. Groups exhibited a difference in responding for 30 μg/kg nicotine, with animals receiving the rapid 1 sec infusion earning nearly double the infusions earned by the group receiving slow 20 sec infusions. Other studies have produced mixed results with moderate increases in infusion duration of nicotine finding either no evidence of self-administration using 6 sec infusions or even a moderate but significant increase in infusions earned by mice receiving 3 sec compared to 1 sec nicotine infusions (Belluzzi et al., 2005; Fowler & Kenny, 2011).

What is clear from the published literature is that there is no consensus, and considerable variability, regarding the impact of infusions in the range of 1 sec (“rapid” infusion), which has been standard in most nicotine self-administration studies, to tens of seconds (“slow” infusion). Discussion of the rate of iv delivery should not simply focus on what rate supports the most robust self-administration. What infusion profile best models nicotine delivery in people depends on the tobacco product or nicotine-delivery system (e.g., electronic cigarette or nicotine patch) being considered. This issue will become increasingly important as new nicotine delivery devices are developed and their abuse liability is evaluated. Additionally, simply comparing rates of nicotine self-administration may miss important differences in underlying mechanism. For example, Sorge and Clarke (2009a) have reported the intriguing observation that dopamine antagonists differentially impact nicotine self-administration depending upon whether nicotine is delivered via “rapid” or “slow” infusions. Likewise, other moderating variables (e.g., cues) may be more or less important depending on the rate of nicotine delivery. Indeed, whereas the primary reinforcing actions of nicotine, along with the associative actions derived from them, require discrete infusions of nicotine contingent on behavior, the non-associative reinforcement enhancement actions of nicotine are observed with slow, even systemic, administration, as well as rapid iv infusions (Caggiula et al., 2009).

3.3 Food restriction

Most self-administration studies of addictive drugs, including nicotine, are conducted on mildly food-restricted rats, with food limited to approximately 80% of the amount of food consumed with unlimited access. This food restriction maintains weight gain, though at a rate below what is observed with unlimited access to food. Experimental animal models of drug abuse use food restriction for a variety of reasons. Constraining growth might prolong patency of the catheter. It might also be argued that animals are not normally exposed to easy access to unlimited food sources and that unlimited access results in overweight and unhealthy animals (Abelson, 1995; Speakman, 2011). Most importantly, food restriction leads to higher levels of responding in operant procedures and contributes to the overall motivational state of the animal (Lang, 1977). Food restriction typically results in more robust self-administration and self-administration of low doses of nicotine that is not observed without food restriction (Corrigall & Coen, 1989; Donny et al., 1998; Singer et al, 1978). This increase in self-administration may simply result from the animals being more likely to explore their environment and respond on operanda, as chronic calorie restriction leads to an increase in physical activity prior to food availability (Duffy, 1990; Russell et al., 1987). Interestingly, there is evidence that calorie restriction also increases cigarette use in human smokers (Cheskin et al., 2005). It is important to emphasize, however, that food restriction is not required for rats to acquire self-administration behavior for nicotine (Donny et al., 1998; Peartree et al., 2012; Yan et al., 2012). Whatever the explanation for why moderate food restriction enhances nicotine self-administration (and responding in other operant procedures as well), the imposition of this ‘motivational state’ is an important variable in nicotine self-administration procedures.

3.4 Session length

The vast majority of nicotine self-administration studies rely on daily (or 5 day per week) sessions lasting an hour or a few hours. Other studies allow more continual self-administration access, allowing rats to respond for nicotine infusions for 22 or 23 h/day with a short break for cleaning the operant chambers. Rats readily self-administer nicotine in both limited and extended access procedures and, importantly, the dose-response relationship for self-administration is similar (Matta et al., 2007). Extended access procedures result in greater daily nicotine intake, although the rate of infusions per hour is reduced. Given the more continuous nature of extended access, animals are more likely to develop nicotine dependence as a results of nicotine exposure that is maintained for a prolonged period each day and may undergo withdrawal if access to nicotine is terminated or pharmacologically precipitated (O'Dell et al., 2007), or even just with significant gaps in self-administration. Consequently, dependence may result in another reinforcing action supporting additional behavior that is not observed in the limited access procedures, negative reinforcement related to withdrawal suppression. Interestingly, Cohen et al (2012) observed that if periods of extended access are spaced with one or two days of no access, the rates of self-administration are substantially greater compared to both extended access without those breaks and limited access sessions with similar days of no access. In contrast, others have argued that limited and extended access paradigms produce comparable levels of dependence (Paterson & Markou, 2004). Despite these potential differences, most studies continue to use limited access because of practical issues and limited access protocols are well suited to examine the reinforcing properties of nicotine in the absence of issues associated with withdrawal.

3.5 Sex and age

The sex of a subject may also moderate iv nicotine self-administration, an important issue to consider given that there are sex differences in tobacco use. Although men are more likely than women to smoke, data from human studies and national surveys of smoking behavior suggest that sex differences are complex and could differentially impact susceptibility to initiate tobacco product use, the progression to dependence, and difficulty with successful cessation (Benowitz & Hatsukami, 1998; Brady & Randall, 1999; Carroll et al., 2004; Kim & Fendrich, 2002; Lynch et al., 2002; Lynch 2006, 2009). These differences may be related to differences in pharmacokinetics and metabolism (Jensvold et al., 1996; Kyerematen et al., 1988), brain development, physiology and function (Berchtold et al., 2008; Chambers et al., 2003), and circulating reproductive hormones (Becker & Hu, 2008; Breslau & Peterson, 1996; O’Hara et al., 1989; Perkins et al., 1999). Women have also been shown to be more reactive to nicotine-associated cues (Perkins et al., 1999; 2001) and stress, which may lead to a greater propensity to relapse (Perkins et al., 2013; Schnoll et al., 2007; Xu et al., 2008).

Sex differences in tobacco use can be recapitulated in rat models of nicotine self-administration suggesting a biological basis. Adult female rats have been shown to respond more for nicotine, although they also demonstrated increased responding on an inactive lever (Chaudhri et al., 2005). In other studies using nicotine paired with a visual stimulus (discussed more below), male and female rats acquired nicotine self-administration at a comparable rate and to similar asymptotic levels within a standard range of nicotine doses (30-90 μg/kg/infusion), but females acquired self-administration faster and reached higher break points on a PR schedule of reinforcement at lower doses of nicotine (Donny et al., 2000; Lanza et al., 2004). In a 23-h extended access test, adult female rats had higher rates of responding for a large dose (60 μg/kg/infusion) than adult males rats (Grebenstein et al., 2013). Finally, females may experience greater withdrawal from nicotine, particularly those related to anxiety/stress (for review see O’Dell & Torres, 2014). However, to our knowledge, sex differences related to negative reinforcement have not been evaluated.

Similarly, the age of a subject may also influence nicotine reinforcement. Approximately 90% of adult daily smokers initiated use prior to the age of 18, and nearly all adult smokers began prior to the age of 25 (CDC, 2012; USDHHS, 2012). National surveys of high school students report that 23% of students are current users of tobacco products (use on at least 1 of the last 30 days) (CDC, 2010). Earlier initiation of smoking increases the likelihood that someone will become a heavy smoker, be more dependent, and have greater difficulty quitting (USDHHS, 2012). These epidemiological data suggest that adolescence represents a period of vulnerability to nicotine use and dependence. Adolescents are biologically driven to seek novelty and risk (Spear, 2000) and are also more reactive to stress and unable to modulate their response in a productive way (Chambers et al., 2003). Although the pre-clinical literature examining developmental differences in nicotine use and reinforcement is limited and varied in approach, collectively the findings suggest that there are differences in the reinforcement of nicotine between adolescents and adults.

Adolescent rats have been shown to self-administer nicotine at rates compared to adults across a range of doses (Chen et al., 2007; Levin et al., 2007, 2011; Natividad et al., 2013). In a direct comparison of adolescent to adult females, adolescent females self-administer significantly more nicotine than adults (Levin et al., 2003). A study testing nicotine self-administration in male and female adolescents found that both sexes responding similarly for nicotine infusions compared to adult rats (Chen et al., 2007); however, female adolescent rats acquired nicotine self-administration more quickly and reached higher rates of responding than adult rats. Lynch (2009) evaluated differences in self-administration of nicotine between male and female adolescents in a long-access paradigm. Again, female adolescents displayed the most robust self-administration behavior under a FR 1 and a PR schedule of reinforcement (Lynch, 2009).

Another issue that comes up in relation to the adolescent period is whether exposure to nicotine during adolescence alters nicotine self-administration in adulthood. Rats exposed to experimenter-administered nicotine as adolescents showed increased nicotine self-administration (Adriani et al., 2003; Natividad et al., 2013) as adults compared to animals only exposed to nicotine during adulthood. These data in rodents support the epidemiological data and hypothesis that early exposure to nicotine may heighten the reinforcing actions of nicotine in adults. However, studies from Shram and colleagues (2008a, 2008b) suggest that adolescence is not a period of enhanced sensitivity to the reinforcing actions of nicotine, noting the lack of a difference in nicotine self-administration between adult and adolescent rats on FR 1 and FR 2 schedules of reinforcement, and greater responding by adults on an FR 5 and PR schedule of reinforcement. Further complicating the influence of age are moderating variables such as stress. When treated with yohimbine, an alpha-2 adrenergic receptor antagonist that many labs use to model a stress response, adolescent females reached significantly higher breakpoints and earned more nicotine infusions than adolescent males at all doses of nicotine tested (7.5, 15, 30 μg/kg/infusion), suggesting greater reactivity after a stressor-like challenge (Li et al., 2014; Zou et al., 2014). These data in rodents support the epidemiological data that female adolescents may be more sensitive to the reinforcing properties of nicotine, particularly with an additional stressor (McKee et al., 2003).

In CPP protocols, adolescent rats and mice have been shown to develop a preference after a single nicotine treatment (adults do not), exhibit a larger degree of preference for nicotine than adults, including at lower doses of nicotine, and lack of an aversion at high doses of nicotine (Belluzzi et al., 2004; Brielmaie et al., 2007; Kota et al., 2007; Torres et al., O'Dell, 2008; Vastola et al.,, 2002). Additionally, animals exposed to nicotine as adolescents showed increased CPP as adults (Adriani et al., 2006).

In sum, the preclinical literature predominantly supports the epidemiological data from smokers that nicotine reinforcement may differ between males and females and that adolescence is a period of heightened susceptibility to the reinforcing effects of nicotine. Additional research utilizing animal models is critical for understanding the biological basis of age- and sex-related differences in nicotine reinforcement. Although these studies can be technically challenging (e.g., catheter patency in young animals; estrous cycle variation), they provide an opportunity for experimental manipulations that are not possible in a clinical setting and therefore can contribute to a better understanding of the causal relationship between development, sex and nicotine reinforcement.

3.6 Other compounds in cigarette smoke

The reinforcing properties of nicotine can be modified by other chemicals in the tobacco product. In cigarettes, nicotine is typically taken along with more than 8000 other chemicals (CDC, 2010). Research on several of the compounds in tobacco smoke have suggested that they may be psychoactive and have abuse potential by themselves and potentiate the abuse liability of nicotine (Hoffman & Evans, 2013). Non-nicotine compounds in cigarette smoke that may have reinforcing value or contribute to the reinforcing actions of nicotine include, but are not limited to: acetaldehyde, minor alkaloids (e.g., nornicotine, myosmine, cotinine, anabasine, anatabine), and ß-carbolines (e.g., harman and norharman). Investigating these smoke constituents, alone and in combination with nicotine, is critical for a better understanding of the reinforcing properties of cigarettes. However, the best methodology for studying the reinforcing potential of these constituents is complicated. For example, given that most of these constituents exist in cigarette smoke at concentrations much lower than nicotine, what doses are appropriate for investigation? Should constituents be investigated in isolation or combined?

Acetaldehyde, one smoke constituent, has received a fair amount attention. It is one of several aldehydes present in tobacco smoke (Houlgate et al., 1989; Xie et al., 2009), resulting from the combustion of polysaccharides, as well as being added in the manufacture of commercial cigarettes. Acetaldehyde is also the major metabolite of ethanol, and has previously been a focus of alcohol abuse liability research (Correa et al., 2012; Deng & Deitrich, 2008). Nicotine and alcohol are often consumed together, adding to the importance of studying the interaction between nicotine and acetaldehyde. Acetaldehyde by itself acts a reinforcer and has been shown to be self-administered orally (Peana et al., 2010; 2011), intravenously (Myers et al., 1982; 1984a; 1984b; Takayama and Uyeno, 1985), and directly into the brain (Amit et al., 1977; Brown et al., 1979; Rodd-Hendricks et al., 2002) by rats. Animals also prefer chambers paired with acetaldehyde in CPP experiments (Melis et al., 2007; Quertemont & De Witte, 2001; Peana et al., 2008; Smith et al., 1984; Spina et al., 2010). Taken together, these results suggest that acetaldehyde has reinforcing properties.

Importantly, acetaldehyde may also interact with nicotine to potentiate the magnitude of reinforcement. Early studies within the tobacco industry examined whether acetaldehyde might potentiate nicotine self-administration. Denoble and colleagues tested the ability of nicotine and acetaldehyde to be self-administered by rats alone or in combination, across a range of doses (0 – 16 μg/kg/infusion for both drugs) with the goal of isolating dose combinations that would produce the highest levels of reinforcement (DeNoble & Mele, 1983). Importantly, acetaldehyde supported higher levels of responding than nicotine at equal doses. The combination of nicotine and acetaldehyde enhanced responding for infusions as compared to responding for infusions of either drug alone. The augmented responding for the combination was most robust at low doses of nicotine (2-8 μg/kg/infusion) and at the highest dose of acetaldehyde (16 μg/kg/infusion) tested. The doses of nicotine and acetaldehyde required to produce maximal responding for iv infusions were quite different from what is in cigarettes, where acetaldehyde concentrations are approximately half that of what is observed for nicotine. More recently, Belluzzi and colleagues (2005) reported that adolescent male rats robustly self-administered a mix of nicotine (30 μg/kg/infusion) and acetaldehyde (16 μg/kg/infusion) while not self-administering either substance individually, which better models the nicotine to acetaldehyde ratio in cigarettes. Interestingly, this interaction between acetaldehyde and nicotine was not observed in adult rats. However, this study used a somewhat unusual set of conditions (i.e., 5.6 sec infusions, ad libitum feeding, FR 1, 3 hr sessions, only 5 self-administration sessions) in which nicotine alone is not self-administered to a significant degree. Taken together, these studies highlight the possibility that acetaldehyde administered along with nicotine can increase the reinforcing properties of nicotine, at least under some conditions.

Although nicotine is the primary alkaloid found in tobacco, accounting for roughly 95% of the alkaloid content, other alkaloids (nornicotine, myosmine, cotinine, anabasine, and anatabine) are also present (Huang & Hsieh, 2007). These minor alkaloids are similar in structure to nicotine and some are metabolites of nicotine (Crooks et al., 1997). A limited body of data suggests that some of these minor alkaloids might have reinforcing properties, but only at doses much higher than or equal to nicotine (Bardo et al., 1999; Caine et al., 2014). In a test of whether rats would self-administer a combination of nornicotine, myosmine, cotinine, anabasine, and anatabine, with doses indexed to their concentration in cigarette smoke relative to nicotine, the alkaloid cocktail did not support self-administration behavior (Clemens et al., 2009). These limited results provide evidence that large doses of some minor alkaloids may have positive reinforcing properties by themselves, but the reinforcing effects of these constituents is likely weak at doses that more closely approximate the levels in tobacco (relative to nicotine). More importantly, this mix of 5 minor alkaloids appeared to enhance the reinforcing actions of nicotine, especially at lower doses of nicotine (Clemens et al., 2009). Using a cued protocol with 4 sec infusions, rats self-administered a solution containing 30 μg/kg/infusion of nicotine along with the minor alkaloids significantly more than just nicotine. The increase in self-administration associated with the co-administration of the minor alkaloids was dependent on the reinforcement schedule (it was observed at FR 5 and PR schedules but not FR 1 or FR 2) and appeared to be larger at smaller doses of nicotine. However, the minor alkaloids co-administered along with nicotine also increased locomotor activity compared to just nicotine, and increased inactive responding on the FR 5 schedule to the same extent as it increased active responding, raising questions as to whether this interaction between minor alkaloids and nicotine results from increased reinforcement. Relatedly, acute systemic treatment with anabasine (20 μg/kg), but not anatabine, nornicotine, myosmine, harman, and norharman, increased the number of nicotine infusions (30 μg/kg/infusion) earned by periadolescent female rats (Hall et al., 2014). However, larger doses of anabasine, anatabine, and nornicotine, when administered systemically prior to nicotine self-administration sessions, suppress the number of infusions (Mello et al 2014; Caine et al 2014; Hall et al., 2014). Although results are limited and mixed, studies like these emphasize the need for increased attention to the interaction between nicotine and other alkaloids that might naturally be consumed along with nicotine.

An alternative approach to examining whether the additional compounds in cigarettes contribute to the reinforcing properties of nicotine in cigarettes is to evaluate self-administration of an extract produced from tobacco or smoke. Recently, Costello et al. (2014) compared self-administration of an aqueous extract of cigarette smoke to that of pure nicotine in adult male rats. At low concentrations of nicotine (3.75 and 7.5 μg/kg/infusion), self-administration was enhanced by the other components in the extract, but self-administration was not different at the highest dose of nicotine tested (15 μg/kg/infusion). While one interpretation of their data is that the other non-nicotine components in their extract enhanced the reinforcing properties of nicotine, it may instead be that these other chemicals in the extract are themselves reinforcing since there was no test of self-administration of a denicotinized extract. Still, self-administration of nicotine in the extract was attenuated by a nicotinic receptor antagonist, suggesting that effects on the nAChR were important for producing the increase in self-administration.

Recently, significant attention has focused on the potential role of MAO inhibition on nicotine reinforcement. This attention derives from clinical studies demonstrating that both MAO A and MAO B are 30-40% inhibited in the brains of smokers relative to non-smokers (Berlin et al., 1995; Fowler et al., 1996a; 1996b). Precisely which constituents account for this level of inhibition is unknown. The β-carbolines, harman and norharman, may contribute to both the inhibition of MAO and the impact on nicotine reinforcement. In a study of the impact of monoamine oxidase (MAO) inhibition on nicotine self-administration, norharman was given chronically to inhibit MAO, resulting in a potentiation of self-administration (Guillem et al., 2006). However, the dose of norharman was substantially higher than that actually delivered in tobacco smoke. Since the constituents in cigarette smoke that lead to MAO inhibition are unknown, several studies in rats have attempted to understand the impact of MAO inhibition on nicotine reinforcement using known MAO inhibitors that are not present in tobacco smoke. Using drugs such as tranylcypromine and phenylzine to inhibit MAO, studies have consistently shown increased self-administration of nicotine, especially at low doses of nicotine (Smith et al., 2014a; Villegier et al, 2007a, 2007b). Three points are worth making here. First, large doses of drugs that inhibit MAO appear to increase the reinforcing properties of nicotine (Smith et al., 2014a; Villegier et al., 2007a, 2007b). Second, the interpretation of these data is complicated by the possibility that the effects produced by large doses of MAO inhibitors may be due, at least in part, to actions of these drugs other than MAO inhibition (Loftipour et al., 2011; Villegier et al., 2007a, 2007b). Third, studies published to date have not examined partial inhibition, as seen in smokers.

As the preceding paragraphs make clear, studying which constituents in addition to nicotine that might impact reinforcement may provide important insight into how nicotine, in the form of tobacco products, reinforces behavior. This work is still in its infancy and the challenge of untangling the role of other constituents can be daunting; namely, it is not clear what constituents should demand experimental focus and how to best model the potential interactions between constituents. However, it is also important to recognize that in the absence of nicotine, these data show these other constituents (in the levels found in tobacco products) are not reinforcing. This observation affirms the conclusions reached over the last several decades that nicotine is the primary addictive substance in tobacco. Hence, these other constituents are still best viewed as potential moderators of the effects of nicotine; they do not appear to be sufficient for maintaining tobacco use behavior.

4.0 Primary reinforcing actions of nicotine

As described up to this point, rats will self-administer nicotine in operant protocols. However, these are typically cued protocols that do not distinguish among primary reinforcement, reinforcement enhancement (discussed below), and the associative processes that may then result over repeated experimental sessions. If there were no external cues that were associated with nicotine delivery, then only the primary reinforcing action of nicotine would be present to support self-administration behavior. The few studies that have examined nicotine without additional associative cues provide support for nicotine acting as a primary reinforcer (Chaudhri et al., 2007; Donny et al., 2003; Palmatier et al., 2006; Sorge et al., 2009b). For example, Sorge and colleagues (2009b) allowed rats to acquire nicotine self-administration (15 μg/kg/infusion; infusions delivered over 30 sec). Though few infusions were earned across the sessions and rats acquired very slowly, acquisition criterion were met without additional cues to support behavior (Sorge et al., 2009b). However, the dose range that supports self-administration is narrow, relatively few infusions are earned, and the rate of behavior does not increase in proportion to changes in the schedule of reinforcement (Caggiula et al., 2002a; Chaudhri et al., 2005, 2007; Donny et al., 2003). Certainly, the bulk of the reinforcing actions of nicotine in typical self-administration procedures cannot be explained as primary reinforcement alone. The next section will begin to dissect the role of environmental cues and their importance for maintaining smoking behavior.

5.0 Associative learning and the influence of stimuli predicting the effects of nicotine

As a consequence of the relatively weak reinforcing effects of nicotine alone, most self-administration procedures use cues paired with nicotine delivery. The use of cues is not an inherent flaw in the rodent model of human smoking, as all nicotine self-administration in humans is cued in some way. These environmental stimuli can function in multiple roles including as conditioned stimuli that trigger conditioned responses, conditioned reinforcers, and discriminative stimuli.

5.1 Conditioned responses to nicotine-associated stimuli

As a result of Pavlovian conditioning, stimuli paired with nicotine can elicit conditioned responses (Pavlov, 1927). A typical Pavlovian conditioning preparation involves an existing reflexive relationship between a stimulus (Unconditioned Stimulus, US) and response (Unconditioned Response, UR). In this case, nicotine (US) results in a wide range of behavioral and physiological changes (UR). Then, an originally neutral stimulus is paired with the presentation of nicotine. Following one or more pairings, the originally neutral stimulus elicits a conditioned response (CR) in the absence of the US, and is now called a Conditioned Stimulus (CS). In this case, any environmental stimulus that is consistently predictive of nicotine may come to serve as a CS and elicit a CR. In an operant procedure, the CS might be a light or a tone paired with nicotine delivery (provided the cue does not initially by itself elicit the response). In the context of cigarette smoking, the look or feel of a cigarette, the taste of tobacco, lighters, the cigarette pack, the smoking corner outside of the office, the effects of alcohol, certain friend groups, etc, might all function to elicit conditioned responses, even in the absence of subsequent nicotine delivery, because they have previously been repeatedly paired with nicotine.

Evidence that stimuli associated with nicotine can elicit a CR come from a wealth of both animal and human literatures suggesting that smoking stimuli increase craving or desire to smoke cigarettes (Conklin & Tiffany, 2002b; Lazev, Herzog, & Brandon, 1999; Wertz & Sayette, 2001). This craving is often considered to be a CR that results from the pairing of these stimuli with nicotine delivery (Conklin & Tiffany, 2002a). Evidence from our laboratory has shown that tolerance to the antinociceptive and corticosterone (CORT) elevating effects of nicotine can be abolished if nicotine is delivered in a novel context (Caggiula et al., 2002b). These data suggest that the tolerance that develops is a result of a conditioned response to the environmental stimuli that reliably predict nicotine delivery. When these environmental stimuli are absent, there is no conditioned response and nicotine has the same effect as the first administration. This study highlights an important issue: while a CR is often similar to the UR, it can sometimes be in the opposite direction, called a compensatory conditioned response (Siegel, 1988). In this example, nicotine (US) results in antinociceptive and CORT elevation (UR). The stimuli (CS) that are paired with nicotine (US) may result in a conditioned response opposite of that effect, so that nicotine delivery produces a smaller change than it would have acutely.

Nicotine-associated stimuli result in CRs that increase the likelihood of engaging in smoking behavior both while nicotine is being actively self-administered and as a trigger to reinitiate nicotine seeking. In relation to the former, it is well accepted that the presentation of cues paired with reward results in increased responding for the reward, even if they were never presented contingent upon the response (Rescorla & Solomon, 1967), a phenomenon known as Pavlovian to instrumental transfer. This phenomenon has received relatively little attention in animal models of nicotine self-administration, but would suggest that the mere presence of CS might facilitate both nicotine-taking and other forms of reinforced behavior. The more common conceptualization of how CS influence nicotine-seeking occurs during abstinence or following extinction when the CS triggers a motivational state or action schema that can lead to the experience of craving (Berridge & Robinson, 2003; Tiffany, 1990). Indeed, cue-elicited craving is reliably linked to smoking in abstinent individuals (Sayette & Tiffany, 2013), although this effect is not clear when assessed in non-abstinent smokers (Perkins, 2009a). Hence, CS can elicit a wide range of CRs including responses that may impact the probability or intensity of nicotine-seeking behavior.

5.2 Nicotine-associated stimuli as conditioned reinforcers

Because smoking stimuli have been paired with nicotine, and nicotine functions as an unconditioned (i.e., primary) reinforcer, these cues can come to reinforce behavior on their own (i.e., become conditioned reinforcers). In a stringent test of this effect, rats with a history self-administering nicotine paired with a CS learned to perform a novel response that was only reinforced by the CS (Palmatier et al., 2007). Likewise, rodent self-administration research has demonstrated that the continued delivery of cues after nicotine has been removed will maintain responding, and this rate of behavior is higher than if cues are also removed (Markou & Paterson, 2009). Clinical research confirms that, over the course of a week or so, smokers will continue to smoke low nicotine content cigarettes (Donny et al., 2007; Donny & Jones, 2009), with moderate to no decrease in smoking behavior. Furthermore, denicotinized cigarettes have been shown to substitute for nicotine-containing cigarettes better than nicotine gum (Johnson, Bickel, & Kirshenbaum, 2004), and the delivery of smoking stimuli has been shown to increase ratings of liking and satisfaction better than iv nicotine (Rose et al., 2000). These data clearly indicate that these cues associated with nicotine can reinforce behavior.

Likewise, clinical experimental studies show that individuals who try to refrain from smoking are significantly more likely to lapse regardless of the nicotine content of those cigarette; even cigarettes with very little nicotine increased the probability of relapse compared to not smoking (Juliano et al., 2006). These data indicate that interacting with smoking stimuli may precipitate smoking behavior during abstinence. The presentation of nicotine-associated cues has also been shown to increase previously extinguished self-administration behavior in experimental animals in a phenomenon known as cue-induced reinstatement (LeSage et al., 2004). Cue-induced reinstatement is very robust; indeed, the magnitude of reinstatement is greater when it is induced by cue presentation than by nicotine (LeSage et al., 2004). Upon cessation of nicotine use, cues should eventually undergo extinction as they no longer reliably predict nicotine delivery (Caggiula et al., 2001; Cohen et al., 2005; Liu et al, 2007, 2008, 2010). However, extinction is context dependent (Wing & Shoaib, 2008; Bouton, 2011), so extinction learning would need to occur in multiple contexts before cues would be fully extinguished, a potentially lengthy process.

5.3 Nicotine-associated contexts as discriminative stimuli

Smoking cues and contexts can also serve as discriminative stimuli, which tend to be broader contextual stimuli, signaling when behavior will result in reinforcement. In a typical discrimination preparation, individuals learn that a behavior will result in reinforcement in the presence of one stimulus, and will not result in reinforcement in the absence of that stimulus (Skinner, 1953). For learning to take place, the probability of reinforcement in the presence of the stimulus must be greater than the probability of reinforcement when the stimulus is absent. In self-administration paradigms, rodents undoubtedly learn that the operant chamber signals the availability of nicotine, although rodents unavoidably spend some time in the chamber when nicotine is not available (right before the session is started, right after the session ends, during time out periods post-infusion when the drug may no longer be available). Some researchers use other stimuli (cue lights, house lights, tones) to signal the start of the session or to signal “time in” (Belluzzi et al., 2005; Grebenstein et al., 2013; Hall et al., 2014). The presence of these stimuli will increase the likelihood of engaging in the reinforced response. This issue is considered in more detail elsewhere in the book.

These contextual stimuli make conducting human laboratory smoking research difficult because smoking behavior has not been previously reinforced in the laboratory context and conclusions drawn from studies conducted in laboratory environments may not extend to the natural environment. In two studies, smokers were asked to smoke cigarettes with very low nicotine contents. In a study conducted in an in-patient hospital unit, there was a decrease in smoking behavior when smokers switched to these low-nicotine cigarettes (Donny et al., 2007). However, smoking behavior did not change when a similar study was conducted in the natural environment (Donny & Jones, 2009). These data parallel the work by Wing and Shoaib (2008) in which extinction was shown to proceed more quickly in a novel environment not associated with previous nicotine self-administration. It may have been easier for smokers to learn that smoking no longer resulted in nicotine delivery in a completely novel context than the natural environment in which smoking behavior had a long history of being reinforced across many contexts (Conklin, 2010; Wray et al., 2011).

5.4 Individual differences in associative learning effects

There is likely a large degree of variability in the degree to which these cues are involved in smoking behavior between individuals. One theory posits that all individuals learn about cues in their environment, but there is variability in the degree to which these cues are “wanted” (Robinson & Berridge, 2000; Robinson et al., 2014). In rodent models, animals that show attraction towards cues have higher break points for cocaine on progressive-ratio schedules (generally considered to be an indicator of motivation to obtain drug) (Saunders & Robinson, 2011), the development of cocaine-paired CPP (Meyer et al., 2011), greater cocaine sensitization upon repeat treatment (Flagel et al., 2008), and greater cue-induced reinstatement (Yager & Robinson, 2010). Future research may provide important insight into the variability observed in nicotine self-administration and the role of cues following nicotine reduction. Sex may be another determinant of cue effects. On average, women are more likely to be affected by smoking cues than men, who may be more directly influenced by nicotine (Perkins, 2009b; Perkins et al., 2002). However, preclinical work does not support this idea. One study evaluated cue-induced and nicotine-primed reinstatement in male and female rats (Feltenstein et al., 2012). There were no sex- or estrous cycle-dependent differences between male and female rats. Still, expanding this line of research is important for cessation, where women may have less success with nicotine replacement therapy because smoking stimuli are more critical in the maintenance of smoking behavior (Perkins et al., 2002).

6.0 Associative learning and the influence of nicotine as a predictor of other reinforcers

A very different line of research investigates the role of nicotine as a cue for other stimuli. For example, rats can learn to respond on one lever for food if they have received an injection of nicotine and respond on another lever if they received a saline injection (Stolerman, 1989). In these experiments, nicotine signals that one behavior will be reinforced and another will not. Relatedly, humans can learn to engage in one response if they receive nicotine nasal spray, and another response if they receive saline spray (Perkins et al, 1994). Researchers have extended this finding to show that nicotine, when paired with a reinforcer, even when it is not contingent upon behavior, can increase the rate of behavior directed at the location of reinforcer delivery (i.e., goal tracking; Besheer et al., 2004).

The role of nicotine as a predictor of other reinforcers in acquiring or maintaining smoking behavior is unclear. However, nicotine delivery is paired with many reinforcing stimuli in the natural environment, and nicotine may function as a CS and a conditioned reinforcer, through this pairing. For example, an adolescent who receives peer acceptance when engaging in smoking behavior may experience nicotine as a conditioned reinforcer because of this repeated pairing over time (i.e., nicotine predicts peer acceptance). It is difficult to show experimentally that nicotine can acquire additional reinforcing value through this association because nicotine has existing primary reinforcing value. However, research has shown that pairing diazepam, an anxiolytic, with money can result in a preference for diazepam, highlighting the ability of drugs to acquire reinforcing value through pairing with other reinforcers (Alessi et al., 2002). Nicotine may also come to elicit conditioned responses as a result of pairing with other drugs of abuse, although there is no existing research in this area. Drugs such as alcohol, marijuana, and caffeine are frequently co-used with nicotine. Smokers who have a cigarette with their morning coffee may associate nicotine with coffee and enjoy cigarettes even on mornings when they choose decaffeinated coffee.

7.0 Reinforcement-enhancing effects of nicotine

The primary reinforcing effects of nicotine and the consequent associative and conditioned reinforcing properties of nicotine are important in driving smoking behavior, but additional non-associative effects of nicotine on reinforced behavior may be equally important. Nicotine potentiates the reinforcing properties of other rewards. This latter effect occurs independent of any predictive relationship between nicotine and either the other stimulus or the target behavior (Caggiula et al., 2009).

In the first study to describe the effect of nicotine on responding maintained by other stimuli in the environment, male adult rats were allowed to respond for nicotine, saline and/or a visual stimulus (VS; the 1-sec onset of a cue light above the active lever and the 60-sec offset of the chamber light) in a between groups design (Donny et al, 2003). Response-continent nicotine, by itself, results in very low rates of responding and, in some cases, failed to support self-administration. Interestingly, responding for the VS, which has been used previously as a drug-cue (Corrigall & Coen, 1989; Donny et al., 1995), supported behavior even in the absence of nicotine, suggesting the VS functioned as an unconditioned reinforcer. This finding is consistent with data describing that sensory stimuli can function as reinforcers (Fowler 1971, Harrington 1963). Importantly, pairing nicotine with the VS produced a synergistic, not just additive, effect on behavior. Responding was more than twice the sum of response rates produced either by nicotine alone or the VS alone. This is consistent with previous studies demonstrating the importance of environmental cues in nicotine self-administration (Caggiula et al., 2001, 2002; Cohen et al., 2005), but also raised questions about the nature of the relationship between nicotine and the VS. The synergistic effects seemed disproportional with the reinforcing properties of the two stimuli. However, this study included a critical control condition that provided a potential answer to these questions. In this group, animals were allowed to respond for the VS while receiving infusions of nicotine that were controlled by (yoked to) another animal. Remarkably, there were no differences in responding for the VS during acquisition between the contingent and noncontingent nicotine conditions. Both contingent and non-contingent infusions of nicotine resulted in the synergistic enhancement of responding for a VS compared to responding for the VS with saline infusion (Donny et al, 2003). Therefore, the increase in responding by the pairing of nicotine with the VS could not be explained by the VS functioning as a conditioned stimulus.

Enhancement of the reinforcement by nicotine is entirely consistent with extensive data on the effects of nicotine, and other drugs of abuse, on intracranial self-stimulation (ICSS) and CPP. In ICSS studies, a stimulating electrode is typically placed in the posterior lateral hypothalamus and rats respond for brain stimulation reward. Systemic (Harrison, 2002) and self-administered (Kenny & Markou, 2006) nicotine lower the threshold for ICSS, indicating that nicotine enhances the rewarding properties of brain stimulation. Likewise, early studies demonstrated that psychostimulants enhance responding for conditioned stimuli (Beninger, 1980; Robbins & Koob, 1978). Although these studies were never extended to nicotine, they highlight that these effects are not unique to nicotine, an observation we have also made (Chaudhri et al., 2006b).

7.1 Reinforcement-enhancing effects of nicotine occur across routes of administration

It is possible that the pattern of nicotine delivery may affect responding for the VS. In particular, yoked infusions of nicotine might result in intermittent unintentional pairings between nicotine and responding for the VS, leading to an associative relationship. To test whether the observed enhancement was due to spurious associations, responding for the VS was tested in rats receiving either rapid non-contingent infusions independent of responding for the VS, a constant nicotine infusion over the 1 h period, or contingent nicotine infusion (Donny et al., 2003). Despite the varied patterns of nicotine delivery, all rats had similar elevated levels of responding for VS presentations compared to saline. Similarly, systemic injection of nicotine administered before the session and even acute treatment with osmotic minipumps enhances responding for VS presentations. These data support the hypothesis that nicotine can directly enhance behavior maintained by unconditioned nonpharmacological reinforcers and that the enhancement is not dependent on a discrete temporal relationship with the stimulus or the behavior. Importantly, replacing nicotine with saline resulted in an immediate reduction in responding to levels similar to controls. Furthermore, reinstating nicotine to contingent and noncontingent groups immediately increased responding to pre-extinction levels (Chaudhri et al., 2007; Donny et al., 2003; Palmatier et al., 2007). These basic observations have now been replicated many times across a range of doses, routes of administration, schedules of reinforcement, and reinforcing stimuli, including conditioned reinforcers (Chaudhri et al., 2005; 2006a; Donny et al., 1999; Palmatier et al., 2006).

To further examine the relative contribution of the primary reinforcing and the reinforcement enhancement properties of nicotine, we utilized a paradigm in which rats had the option to respond for nicotine and the VS independently (Palmatier et al., 2006). The primary group of interest pressed one active lever for the presentation of the VS and another lever for infusions of nicotine. A separate group of rats (nicotine + VS) had standard self-administration operanda: one active lever controlled both the presentation of the VS and nicotine infusion. Control groups received either nicotine infusions (nicotine only) or VS presentations (VS only) contingent upon behavior on the active lever (the other lever was inactive). Nicotine alone and VS alone maintained relatively moderate levels of behavior and access to both reinforcers on one lever resulted in synergistic enhancement of responding, as we had previously reported. Surprisingly, when rats had access to each reinforcer on a separate lever, responding on the nicotine lever was low, about equivalent level to that of the nicotine only group. However, responding for the VS lever was enhanced by the same magnitude as the nicotine + VS group. Additional studies from our lab have demonstrated that the metabotropic glutamate 5 receptor antagonist MTEP can suppress responding for nicotine alone, with no effect on the reinforcement enhancing properties of nicotine, indicating that the primary and enhancing reinforcement properties of nicotine can be pharmacologically dissociated (Palmatier et al., 2008). To our knowledge, there are no data indicating what neural mechanism(s) may be responsible for the reinforcing enhancement properties of nicotine. These collective studies indicate that the reinforcement enhancing properties of nicotine are potent even when only a small amount of the drug is administered that the effects are behaviorally and pharmacologically dissociable, and that the high rates of self-administration observed in the paired group is likely due to the reinforcement enhancing effects of nicotine (Palmatier et al., 2006,2008).

Additional studies by Chaudhri et al (2006a) demonstrated that in rats that acquired self-administration across a range of nicotine doses in the absence of a pairing with a nonpharmacological stimulus, the addition of the VS resulted in an immediate and robust increase in responding. These results are important because they 1) replicate the observation that rats will acquire responding for nicotine self-administration without co-incident nonpharmacological stimuli, but that this effect depends on larger doses of nicotine, and 2) demonstrate that the effect of the addition of the VS was most prominent at low doses of nicotine.

Taken together, these studies emphasize why responding for nicotine infusions is so robust in the presence of a non-drug stimulus: it is not the nicotine per se but the synergistic interaction between nicotine and nonpharmacological stimuli that produces a significant increase in behavior. Environmental stimuli with reinforcing value paired with nicotine increase the rate of acquisition of nicotine self-administration and the rate of maintained self-administered behavior independent of the route and speed of nicotine delivery (Caggiula et al., 2009; Chaudhri et al., 2006a).

7.2 Reinforcement enhancing effects of nicotine occur across age and sex

Until recently, the majority of research on the reinforcement enhancing effects of nicotine has been conducted in adult male rats. However, the enhancing effects have been demonstrated in adolescent male rats as well. Work from our laboratory assessed whether responding for the VS was enhanced by exposure to nicotine in adolescent (postnatal day 29-42), male rats (Weaver et al, 2012). Like adults, adolescent rats responded for presentations of the VS and subcutaneous (sc) nicotine just prior to the session increased responding for the VS. The effect was qualitatively similar to that observed in adults at the dose tested. Similarly, adolescent male rats (P39-40) tested in a CPP procedure using social reward (i.e., a “playmate”) found that nicotine increased the amount of time spent on the side of the chamber paired with a social playmate (Theil et al, 2009). To date, potential developmental differences in the reinforcement-enhancing effects of nicotine have not been thoroughly examined.

Another important question given that female rats acquire nicotine self-administration more quickly and reach higher breakpoints on a PR schedule when nicotine is paired with the VS (Donny et al., 2000; Lanza et al., 2004), is whether there are sex differences in the reinforcement-enhancing effects of nicotine. To our knowledge, this question has not been directly addressed. Studies suggest, however, that there are sex differences in self-administration of both nicotine alone and in combination with the VS. In a study by Chaudhri et al (2005), animals first acquired responding for contingent nicotine infusions without additional nonpharmacological stimuli. Females earned more infusions of moderate to large (60-120 μg/kg) doses of nicotine than males. Then the VS was added with active lever presses resulting in delivery of both the nicotine infusion and the VS. The addition of VS caused a doubling in responding in both sexes when animals were self-administering 30 or 60 μg/kg/infusion (Chaudhri et al., 2005). A separate group of male and female rats were allowed to respond for VS presentations without a nicotine infusion and confirmed that the VS functioned as a primary reinforcer in females as well as males, suggesting the potentiation in self-administration may have been driven by the reinforcement-enhancing effects of nicotine in both sexes.

7.3 Reinforcement-enhancing effects of nicotine occur in humans

The ability of nicotine to enhance reinforcement for other stimuli is well established in the rodent model, but whether reinforcement enhancement is important in human smokers has received less attention. Studies to date, however, are consistent with the animal literature. Smoking increased ratings of attractiveness (Attwood, 2009) and reported feelings of happiness during happy films (Dawkins, 2007). Likewise, transdermal nicotine increases the response bias towards a rewarded stimulus (Barr et al., 2008). Furthermore, abstinence from smoking reduces the blood oxygen level dependent response to monetary rewards in the caudate (Sweitzer et al, 2013). Finally, in the most direct study of reinforcement-enhancing effects in humans, Perkins and Karelitz evaluated the ability of nicotine to enhance operant responding for a variety of rewards in smokers (Perkins & Karelitz, 2013a). Participants were dependent and non-dependent smokers that were deprived of nicotine at the start of the study and asked to respond on a PR schedule adapted for human subjects for a designated music reward. Smoking was able to enhance responding for high preference music, as compared to no smoking and smoking at low levels that are sub-threshold for enhancement (Perkins & Karelitz, 2013a). The implications of these results are discussed in detail below, but they provide clear evidence that nicotine enhances reinforcement in humans.

7.4 Enhancement is moderated by the type and nature of the reinforcer

If a stimulus has little or no reinforcing value (i.e., neutral), nicotine should have no effect on enhancing the rewarding properties of that stimulus unless it gains conditioned reinforcing value (as discussed above). In a study designed to test this hypothesis, we compared the effects of systemic noncontingent injections on two sensory stimuli that differed in their unconditioned reinforcing effects (Palmatier 2007). During sessions of acquired stable behavior, rats responded significantly more for a houselight-off stimulus (5-sec extinction of the houselight paired with an 83 dB tone) than a lever light-on stimulus (5-sec onset of a stimulus light above the active lever paired with an 83 dB tone), indicating that the houselight-off stimulus has higher incentive value. Saline injections had no effect on behavior. Nicotine caused an increase in responding for the houselight-off stimulus, but with no effect on the lever light-off stimulus. These results support the notion that nicotine has the ability to non-associatively enhance responding only for stimuli with reinforcing value.

More recent studies have further confirmed our hypothesis that the magnitude of the nicotine-induced enhancement is modulated by the strength of the reinforcer. The quantitative relationship between the enhancing effects of nicotine and the incentive value of the reward was tested using rats responding on a PR schedule of reinforcement for the delivery of liquid sucrose (0, 5, 20, and 60%; Palmatier et al., 2012). Systemic, noncontingent injection of nicotine enhanced the responding for sucrose reward. The breakpoint reached (final number of responses required for reward delivery) was potentiated by nicotine, with the magnitude of enhancement increasing with increasing sucrose concentration. These results are particularly interesting, as they raise the possibility that the reward-enhancement effects of nicotine may override other pharmacological actions of nicotine, as nicotine is known to be a potent appetite suppressant. In fact, when the sucrose solutions (2.5%, 5%, and 10%) were available on a reinforcement schedule that demanded low responding rates for each reward delivery (FR 3), nicotine reduced responding. As has been previously suggested, the effects of nicotine may dependent on the schedule of reinforcement because nicotine has multiple actions on food reinforcement; nicotine may enhance satiety when food is relatively freely available and enhance the reinforcing efficacy of food when it is relatively restricted (Donny et al., 2011).

Work from our lab has also established that nicotine has the ability to enhance responding for conditioned reinforcers. Work by Chaudhri and colleagues (2006) used a light-tone stimulus as a CS predictive of sucrose pellet delivery. Both contingent and noncontingent delivery of nicotine enhanced responding for a light-tone stimulus when it had been previously paired with sucrose, but not if the stimulus wasn’t predictive of sucrose. Extending this work to a nicotine-associated CS, Palmatier and colleagues (2007), trained animals to lever press for nicotine paired with as unconditioned light stimulus. Animals were then allowed to nosepoke for the CS alone and demonstrated the predicted conditioned reinforcing effects (as described above). Interestingly, noncontingent delivery of nicotine further increased the rate of responding for the CS (Palmatier et al., 2007), confirming that non-contingent nicotine can enhance responding that is maintained solely by a nicotine-associated conditioned reinforcer.

Work investigating the ability of nicotine to enhance reward in humans has found similar results. One study tested if smoking enhanced responding for music reward on a PR schedule (Perkins & Karelitz, 2013a). Participants were instructed to bring a pack of their own cigarettes and a music album of their choice. Different music was designated as high, moderate, or low reward based on the participants own rating of the music on a 0-100 visual analog scale (VAS). Participants completed three 2 hr sessions, each after overnight abstinence, with different levels of smoking prior to assessing responding to hear different segments of music. As our hypothesis would predict, smoking enhanced responding and this was only observed when music was preferred (Perkins & Karelitz, 2013a). Minimal (couple puffs) or no smoking had no effect on behavior. Hence, the effect of nicotine on reinforced behavior in humans depends both on the level of smoke exposure and the degree to which the reward is preferred.

Furthermore, in a separate study investigating that ability of nicotine to enhance responding for reward in humans, smoking after abstinence was able to enhance responding for a preferred music reward, but not for monetary reward (Perkins & Karelitz, 2013b). These data suggest that the reinforcement-enhancing effects of nicotine differ depending on reinforcer type (e.g., more apparent with “sensory” reinforcers). However, it is possible that the small monetary rewards (participants earned an average of about $1.20 in a 2 hr session) were effectively neutral and thus, not able to be enhanced by nicotine.

Together, these studies provide evidence that the reinforcement-enhancing effects of nicotine are dependent upon the reinforcer being presented (Barret & Bevins, 2013; Palmatier, 2012; Perkins & Karelitz, 2013b). Neutral or mild reinforcers as less likely to be impacted by nicotine unless the value of those reinforcers is increased as a consequence of being paired with another reinforcer. In addition, some reinforcers may be more prone to the reinforcement-enhancing effects of nicotine or be masked by other effects of nicotine in some conditions, for example, “sensory” reinforcers and food, respectively.

7.5 Implications of the reinforcement-enhancing effect of nicotine for tobacco and other nicotine-containing products

In comparison to the primary reinforcing actions of nicotine, the reinforcement enhancement actions of nicotine are seen with lower doses and drug delivery that is not temporally tied to behavior. Thus, for example, patch delivery of nicotine might enhance other reinforcers (and may contribute to its efficacy as a smoking cessation aid) but is unlikely to have primary reinforcing actions (Barr et al., 2008). Similarly, new tobacco products that provide nicotine delivery that is neither rapid nor discrete will likely favor the reinforcement enhancement actions of nicotine as opposed to the primary reinforcing actions. E-cigarettes, for example, are perceived favorably by young adults, especially when appealing flavorants are added to the product (Choi et al., 2012). Given the ability of nicotine to enhance responding for sucrose reward (Barret & Bevins, 2013; Palmatier et al,, 2012; Schassburger, 2013), it is possible that nicotine by “vaping” an e-cigarette might enhance the reinforcing properties of the flavorant within that e-cigarette.

Conversely, understanding the potentially different neuropharmacological mechanisms underlying the primary reinforcing and the reinforcement enhancing actions of nicotine may impact the development of smoking cessation pharmacotherapies. Interestingly, both bupropion (Caggiula et al., 2009) and varenicline (Levin et al., 2012), both FDA-approved prescription pharmacotherapies for the treatment of smoking cessation, can substitute for the reinforcement-enhancing effects of nicotine, which may partially underlie their efficacy as pharmacotherapies. To test the ability of buproprion to enhance the reinforcing valence of VS presentations, nicotine or bupropion was systemically administered across sessions (Palmatier et al., 2009). Both nicotine and bupropion increased responding for VS presentations. As expected, the nicotine-enhancement was abolished by administration of mecamylamine, a non-selective nicotinic antagonist. Mecamylamine had no effect on the enhancement caused by buproprion; the bupropion-induced enhancement was blocked by the administration of an alpha adrenergic receptor antagonist, indicating that the reinforcement enhancing effects of these drugs are pharmacologically dissociable. Levin et al (2012) tested the effects of systemic administration of varenicline, a partial nicotinic agonist, on responding for VS presentations with and without co-administration with nicotine (Levin et al., 2012). Varenicline dose-dependently increased VS presentations earned, as well as suppressed nicotine-induced enhancement. As current over-the-counter nicotine replacement therapies are largely ineffective tools in supporting smoking cessation (Kotz et al., 2014), it might be more beneficial to target the development of new medications at the reinforcement enhancing actions of nicotine, as that seems to be a primary mechanism for nicotine self-administration.

8.0 Summary and Conclusions

Nicotine reinforcement is remarkably complex and requires understanding of both associative and non-associative mechanisms well beyond the primary reinforcing effects of the drug itself. From an associative perspective, environmental stimuli that predict nicotine delivery can become conditioned stimuli eliciting conditioned responses that increase the probability of smoking. They will also become conditioned reinforcers through their pairing with nicotine, an unconditioned reinforcer, and will reinforce smoking behavior on their own. Contextual stimuli can become discriminative stimuli, increasing the probability of engaging in smoking behavior. Nicotine can also enter into associative relationships through its pairing with other reinforcing stimuli in the environment and consequently function as a conditioned stimulus or as a conditioned reinforcer, both of which may increase the likelihood of engaging in smoking behavior.

From a non-associative perspective, nicotine, like other psychostimulants, can directly impact reinforcement from other stimuli in the environment. This effect is particularly robust with nicotine and has been emphasized in the “dual-reinforcement” model, which posits that nicotine maintains self-administration behavior as a primary reinforcer and a reinforcement enhancer (Caggiula et al., 2009; Donny et al., 2003). The reinforcement enhancing properties of nicotine have been observed in experimental animals across age and sex, and, more recently, confirmed to impact behavior in humans. Responding for both unconditioned reinforcers (sensory stimuli, food reward) and conditioned reinforcers (nicotine- and nonnicotine-related) can be enhanced by nicotine. The magnitude of the enhancement is dependent on the magnitude of the reinforcer and potentially the type of reinforcer and conditions under which it is available.

In sum, self-administration of nicotine in humans and the rodent model is sustained by three main actions: 1) nicotine acts as a relatively weak primary reinforcer; 2) nicotine can establish conditioned reinforcers in the environment through associative processes; and 3) nicotine can potentiate the incentive valence of other stimuli with reinforcing value. Other actions may also be important (e.g., nicotine as a CS), but future studies will be needed to confirm these effects. These studies confirm that nicotine is the key psychoactive determinant of tobacco product use; however, it is much more insidious than might be expected. Thus, understanding how nicotine acts to maintain behavior is still at the heart of reducing the burden of tobacco dependence.