Counterbalancing in Smoking Cue Research: A Critical Analysis

Abstract

Introduction:

Cue exposure research has been used to examine key issues in smoking research, such as predicting relapse, testing new medications, investigating the neurobiology of nicotine dependence, and examining reactivity among smokers with comorbid psychopathologies. Determining the order that cues are presented is one of the most critical steps in the design of these investigations. It is widely assumed that cue exposure studies should counterbalance the order in which smoking and control (neutral) cues are presented. This article examines the premises underlying the use of counterbalancing in experimental research, and it evaluates the degree to which counterbalancing is appropriate in smoking cue exposure studies.

Methods:

We reviewed the available literature on the use of counterbalancing techniques in human smoking cue exposure research.

Results:

Many studies counterbalancing order of cues have not provided critical analyses to determine whether this approach was appropriate. Studies that have reported relevant data, however, suggest that order of cue presentation interacts with type of cue (smoking vs. control), which raises concerns about the utility of counterbalancing. Primarily, this concern arises from potential carryover effects, in which exposure to smoking cues affects subsequent responding to neutral cues.

Conclusions:

Cue type by order of cue interactions may compromise the utility of counterbalancing. Unfortunately, there is no obvious alternative that is optimal across studies. Strengths and limitations of several alternative designs are considered, and key questions are identified to advance understanding of the optimal conditions for conducting smoking cue exposure studies.

Introduction

Both clinical and experimental studies link cigarette craving to addiction (Killen, 1997; Piasecki, Jorenby, Smith, Fiore, & Baker, 2003; Shiffman et al., 1997; Waters et al., 2004). Much of this research has been conducted in laboratory settings, in which cravings are provoked by exposing participants to smoking-related cues. Cue exposure paradigms have been used to study a diverse set of issues pertaining to nicotine addiction. These include investigating the neurobiology of nicotine addiction using brain imaging methods (e.g., Brody et al., 2004; McClernon, Hiott, Huettel, & Rose, 2005; Wilson, Sayette, Delgado, & Fiez, 2005), testing the efficacy of smoking cessation medications (e.g., Niaura et al., 2005; Shiffman et al., 2003; Tiffany, Cox, & Elash, 2000), and evaluating psychiatric disorders that are comorbid with nicotine dependence (e.g., Fonder et al., 2005). It is clear that cue exposure research is tackling an ever-expanding range of topics.

Despite this burgeoning interest, there has been insufficient focus on key methodological aspects of smoking cue exposure research (Sayette, Shiffman, Tiffany, Niaura, Martin, & Shadel, 2000). Much still remains unresolved about the optimal approaches for manipulating smoking cues (see Tiffany, Carter, & Singleton, 2000). (Generally, cue reactivity refers to reactions to smoking cues after accounting for reactions during control cue exposure [Carter & Tiffany, 1999].) The purpose of this review is to focus on a key element of these studies, namely, the order in which smoking and control cues are administered to participants. Specifically, we evaluated the oft-used approach of counterbalancing the administration of smoking and control cues. Virtually, all cue reactivity studies must address this issue, and we believe that the decision about how best to proceed deserves greater scrutiny and deliberation than it has received.

Researchers conducting cue exposure studies generally recognize the importance of provoking robust reactions to the smoking cues. Studies have employed a range of stimuli, including in vivo cues (e.g., participants hold a lit cigarette), smoking imagery, visual images, interactive virtual reality methods, and administration of stressors. In some cases, participants receive a single exposure to the drug cue and a single exposure to a neutral or control cue, while other studies use multiple trials of both smoking and control cues. (We discuss below number of cue trials.) A critical methodological question concerns the sequencing of the smoking and neutral cues.

It appears to be axiomatic that any cue reactivity study worth its salt would counterbalance the order in which drug and control cues are administered (see Carter & Tiffany, 1999). Indeed, the majority of smoking cue studies we reviewed for this paper counterbalanced the order of cues without feeling any need to justify this design. Presumably, a fixed order (e.g., neutral cue always preceding cigarette cue) would create a confound, such that drug cue effects cannot be disentangled from order of cue presentation. Thus, extraneous factors (e.g., fatigue, habituation, increased withdrawal as time since last cigarette increases across the study) are inadvertently introduced into the study design. If exposure to one cue affects responding to the next cue, then counterbalancing may be needed to control for this order effect.

At first glance, the decision to counterbalance seems obvious. Often ignored, though, is the assumption implicit in counterbalancing procedures that initial cue presentations do not contaminate or bias responses to subsequent presentations (i.e., produce carry over effects) differentially across the various experimental treatments (Hutchison, Niaura, & Swift 1999). Stated differently, counterbalancing is problematic if carryover effects interact with the different experimental treatments or orders of treatments (Keppel, 1982). Yet in some fairly common cue exposure procedures, it is unclear that each order of cue presentation is similarly vulnerable to carryover effects. The possibility of differential carryover effects may be a real concern. Indeed, many prominent research methodologists have concluded that differential carryover effects across conditions preclude use of a counterbalanced design (e.g., Keppel, 1982). Winer (1971) warns of the harmful effects of counterbalancing in his classic research design text: “A strong word of warning is required in connection with order (or sequence) effects … .If such effects exist, randomizing or counterbalancing does not remove them; rather such procedures completely entangle the latter [order effects] with treatment effects” (p. 517). This warning is reinforced by Campbell and Stanley (1963), who note that successful counterbalancing depends on the ability to rule out such interactions.

At issue is whether there is potential for differential carryover effects in smoking cue reactivity studies. There are two different kinds of order interactions that may signal concern for smoking researchers. The simpler one is that the two types of cues (smoking and control) do not exert reciprocal effects on each other. It is possible, for example, that exposure to a smoking cue biases response to a subsequently presented control cue to a greater degree than does exposure to a control cue bias responding to a subsequent smoking cue. That is, the initial smoking cue may elevate cravings, and these cravings do not completely dissipate before the next cue (in this case–control cue) is presented. It therefore would follow that responding to a smoking cue would be fairly unbiased regardless of whether it was presented before or after a control cue. In contrast, responses to a control cue would be uncontaminated only if it were presented prior to exposure to a smoking cue. Accordingly, there is an interaction between order of cues and time such that only the control cue is affected by order, thereby reducing the probability of detecting a cue effect. An example of this pattern is illustrated in Figure 1 and depicts the pattern observed in one of the first studies in this literature (Rickard-Figueroa & Zeichner, 1985).

Figure 1.

Illustration of a cue type by order of cue presentation interaction. There is a carryover effect in the bottom row only, in which a smoking cue affects responding to a subsequently presented control cue.

A second type of interaction that would undermine use of counterbalancing occurs when the order of smoking and control cues are moderated by particular individual difference (e.g., heavy smokers vs. light smokers) or situational (e.g., nicotine deprived vs. nondeprived) factors in the study. One possibility is that the carryover pattern depicted in Figure 1 would be especially pronounced for groups of subjects in whom reactions to smoking cues are strongest (e.g., among heavy nicotine-deprived smokers; Sayette, Martin, Wertz, Shiffman, & Perrot, 2001). Recent reviews suggest that substance abusers show greater attentional processing of drug cues than nonusers (Cox, Fadardi, & Pothos, 2006; Field, Munafo, & Franken, 2009). It also appears that nicotine deprivation increases the degree to which responses to smoking cues become more coherent across response systems (Sayette et al., 2003) and that under conditions of strong cravings, attentional bias toward smoking cues become correlated with strength of urge ratings (see Field et al., 2009; Sayette et al., 2001). Accordingly, carryover effects may be especially likely in nicotine-deprived heavy smokers being first exposed to a smoking cue followed by a control cue, creating just the type of order interaction noted above that raises concern about counterbalancing and that may weaken sensitivity to detect cue effects.

Because we believe that such interactions involving order of cue presentation may be common in the smoking cue exposure literature, it would be useful to review this growing research field to examine how cue counterbalancing has been implemented. We investigated the following questions:

1. Do data exist to suggest that carryover effects are important to address in smoking cue exposure studies that have counterbalanced order of cue presentation?

2. How often have smoking cue exposure studies counterbalanced the sequence of smoking and control cues?

3. How often did they report whether order of cues exerted a main effect on cue reactivity or interacted with other factors to affect cue reactivity? How often was order of cue presentation included as a factor in the analyses?

We first review research relying on a diverse set of measures other than self-reported craving to examine the likelihood that exposure to a smoking cue might produce a carryover effect. While cognitive theories of attention and memory suggest that a smoking cue should create carryover effects on subsequently presented control stimuli (see Field et al., 2009), it remains important to confirm empirically that this is the case. We believe these cognitive performance data raise questions that pertain to studies that have relied on self-reported measures of craving. Next, we review studies that counterbalanced the order of smoking and control cues within a single session as well as studies that counterbalanced the order of smoking and control cues across multiple sessions. Because self-reported urge is the most common index of craving across studies and has shown the most powerful effects (Carter & Tiffany, 1999), we focus on this measure. We conclude with some thoughts about the implications of these findings for conducting smoking cue exposure research.

Both formal and informal approaches were used to search the smoking cue exposure literature. First, we examined the smoking cue studies included in several reviews (e.g., Carter & Tiffany, 1999; Sayette et al., 2000). Additional papers were identified using computer database searches (PsycLIT and Medline) as well as through inspection of reference lists from published journal articles and books. The general criteria for inclusion in the review sample were as follows: (a) the study included a sample of smokers, (b) self-reported urge was assessed during a cue exposure manipulation, and (c) the cue exposure manipulation included both control and smoking cues. Studies were excluded from the review if participants were permitted to smoke during the cue exposure manipulation (as degree of nicotine deprivation also is changing throughout the procedures) or if the study used a between-subjects design for cue exposure.

Evidence for Carryover Effects

Despite the popularity of counterbalancing in smoking cue studies, it is notable that research in and out of the addictions field suggests that drug cues may produce a response that may linger long enough to affect reactions to subsequent cues. Baker, Morse, and Sherman (1987) proposed that drug cues activate well-articulated urge networks, and under certain conditions, it seems hard to imagine that these networks, once activated, would immediately deactivate seconds after a cue presentation. More recent models of craving reinforce the idea that cue-elicited urges may not always disappear completely prior to the next cue presentation (see Kavanagh, Andrade, & May, 2005). Although smokers report (retrospectively) that their craving episodes typically last 6–10 min before dissipating (Heishman, Singleton, & Moolchan, 2003), real-time data collected in the laboratory by Heishman et al. (2004) reveal that cravings actually persist at least 15-min postcue (15 min being the longest time interval assessed in the study) compared with control exposure. In our own laboratory, we find that urge ratings remained elevated for more than 40 min after subjects extinguished a lit cigarette (an in vivo smoking cue exposure; Sayette, Loewenstein, Kirchner, & Travis, 2005). More research is needed to determine the factors that influence the duration of a cue-elicited craving state. For instance, are carryover effects that may appear milliseconds after a cue presentation (e.g., assessed using electrophysiological measures) qualitatively similar to a high urge rating that persists 15 min after cue exposure? Dissipation of the response to the initial cue likely depends in part on the manipulations and measures used in the particular study.

The smoking Stroop task offers a useful paradigm for evaluating potential carryover effects. In a smoking Stroop task, subjects view a series of words, some of which relate to smoking, presented in one of four colors. The task requires subjects to name the color of the word as fast as possible while ignoring word content. Research shows that the latency to name the color increases for smokers when the word is related to smoking (see Waters & Sayette, 2006). Most pertinent to the present review, several studies have found that exposure to salient cues (e.g., drug-related words for individuals who are drug dependent) affect response time on subsequent trials (McKenna & Sharma, 2004; Waters, Sayette, Franken, & Schwartz, 2005; Waters, Sayette, & Wertz, 2003). That is, the attention-demanding effects elicited by exposure to salient cues carryover to affect processing of the next cue. Waters et al. (2003, Studies 1 and 2), for example, found that smokers showed greater interference to words appearing after smoking-related items than to words appearing after neutral items. These findings suggest the first of the two troubling interactions noted above, namely, the existence of a cue type by order interaction such that smoking cues but not control cues, produce carryover effects. Subsequent work replicated these carryover effects in other versions of the “emotional” Stroop paradigm. Heroin users, but not nonusers, responded slower to words appearing after heroin-related words than to words following neutral items (Waters et al., 2005, Study 1). These interactive effects with drug users also generalized to situational manipulations, as subjects anticipating making a stressful self-disclosing speech about their physical appearance responded slower to words following speech stressor-related words than to words following nonspecific stress words or neutral words (Waters et al., 2005, Study 2). In this study, the carryover effects did not persist when there were two neutral words following the smoking trial (Waters et al., 2005).

These Stroop data provide consistent support for the existence of carryover effects on attentional processes (see also Frings, Englert, Wentura, & Bermeitinger, 2010; Kunde & Mauer, 2008). A recent meta-analysis of emotional Stroop studies reinforces this conclusion (Phaf & Kan, 2007). Research also reveals carryover effects of smoking cues on working memory. Wilson, Sayette, Fiez, and Brough (2007) presented smokers with smoking and neutral cues in a counterbalanced fashion before administering a working memory task and reported an order by cue interaction. Performance during the neutral cue was disrupted if subjects had just previously been exposed to the smoking cues (compared with when the control cue exposure came first). As noted by Wilson et al. (2007), “performance on a wide variety of cognitive tasks is likely to exhibit similar [carryover] effects.” (p. 617). Similar effects in other response domains suggest that carryover effects may be widespread. Prior exposure to drug-associated stimuli appears to influence neurobiological responses under purportedly neutral conditions (Breiter et al., 1997). As noted by Franklin et al. (2007), “BOLD event-related paradigms are challenged in this regard, as smoking and nonsmoking stimuli are interspersed across one scanning session, blurring stimulus signals as the differential between them is reduced.” (p. 2308).

In sum, there is converging evidence that carryover effects emerge when smoking stimuli precede presentation of control stimuli. In contrast, when control cues are presented first, there does not seem to be a commensurate carryover effect on subsequent smoking cues. Many theories of craving emphasize the redistribution of nonautomatic processing or attentional resources (Kavanagh et al., 2005; Sayette, 1999; Tiffany, 1990), and as noted above, there is evidence suggesting a link between performance measures of cognitive processing and self-reported urge (Field et al., 2009; Sayette et al., 2001). Thus, these cognitive processing findings reviewed above suggest that carryover effects also may appear when using more traditional self-report measures of craving, which are reviewed below. This interaction raises issues for the use of counterbalancing in smoking cue studies.

There also are data that relate, albeit indirectly, to the second type of interaction that calls into question counterbalancing procedures in cue exposure research. Specifically, certain individual difference and situational factors that moderate cognitive reactions to smoking cues provide suggestive evidence of this concern. Returning to the smoking Stroop literature, some (though not all, see Waters and Sayette, 2006) studies have found that nicotine deprivation increases attentional bias. Research also suggests that perceived smoking opportunity affects smoking Stroop performance (Wertz & Sayette, 2001), as does current interest in abstaining from smoking (Cane, Sharma, & Albery, 2009). In addition, studies find increased attentional bias on the smoking Stroop task for heavy users compared with lighter users (Zack, Belsito, Scher, Eissenberg, & Corrigall, 2001), a finding that also appears in studies using a visual probe task (see Waters & Sayette, 2006). Moreover, as noted above, there is evidence that other types of substance users show differential carryover effects compared with nonusers (Cane et al.; Waters et al., 2005).

Although not all these studies explicitly examined carryover effects, they are consistent with the view that increased focus on smoking stimuli may lead to enhanced carryover effects. Consequently, cue exposure studies that manipulate factors, such as nicotine deprivation or smoking availability or contrast heavy and light smokers, may be vulnerable to differential carryover effects and thus may be poor candidates for counterbalancing. In their recent meta-analysis of the addiction Stroop task, Field et al. (2009) conclude: “We interpret the presence of these carryover effects as indicative of a slow cognitive process that persists even after the substance-related cues have been removed from the stimulus display. Therefore, Stroop interference is likely to reflect the delayed disengagement of attention.” (p. 593). If Field et al.’s well-reasoned assessment is accurate, then it is plausible that a range of smoking cues presented prior to a neutral cue would still affect subsequent craving.

Counterbalancing in Smoking Cue Exposure Studies

Smoking cue studies differ regarding the number of smoking and control cues presented to subjects. Some studies present a single smoking cue, while others administer multiple trials of smoking and control cues. The former approach seeks to provoke a powerful “one-shot” response that lasts throughout the assessment period. Implicit in this single-exposure approach is the concern that such a robust manipulation (e.g., asking subjects to hold, look at, but not smoke a lit cigarette of their own brand [e.g., Sayette et al., 2001]) cannot be repeated again and again in a manner that would be experienced uniformly across trials. The latter approach seeks to the improve reliability, and thus the power, of the manipulation by providing multiple trials of cigarette and control cues (Tiffany et al., 2000). In the next two sections, we address the use of counterbalancing in single-trial and multitrial smoking cue exposure studies.

Single-Trial Studies

Table 1 lists single-exposure studies that counterbalanced the order of control and smoking cues. (We did not include in Table 1 the laboratory studies that used more than two smoking and control cues [e.g., Morissette, Palfai, Gulliver, Spiegel, & Barlow, 2005; Taylor, Harris, Singleton, Moolchan, & Heishman, 2000; Tiffany & Drobes, 1990]. As we note below, it becomes more challenging to test for possible carryover effects as the number of cue presentations [and thus presentation orders] increases. None of these studies reported testing for order effects.) In most cases (see upper portion of Table 1), the order of cue presentation was counterbalanced within a single session. Studies also are included that counterbalanced order of cues across multiple sessions (see lower portion of Table 1). Of the 15 articles (some reporting findings from more than one experiment) included in the upper portion of Table 1, it is notable that 10 did not report testing for a cue by order interaction (e.g., as illustrated in Figure 1), and 6 of these 10 also failed to report even testing for a main effect of order. Among those that did address order, in some cases, there is no mention of testing for interactions of order with the other variables tested (e.g., perceived availability of smoking) that, as noted above, are critical for evaluating counterbalancing. Among the five studies that did address order effect interactions, four reported significant order effects for urge ratings (while the fifth study reported significant order effects for affect ratings). For example, Rickard-Figueroa and Zeichner (1985) counterbalanced the order of exposure to in vivo smoking cues (S) and a nonexposure (NE) condition. They found a Condition × Order interaction for urge ratings: nonexposure ratings differed depending on whether they preceded or followed smoking cue exposure, whereas smoking cue ratings were not affected by nonexposure cues presented in advance. These studies indicate that the assumption that order effects will not interact with drug cue or other key variables is not supported.

Table 1.

Single-Exposure Studies That Counterbalanced the Order Cues

Study	Type of cues	Time interval between cues	Test for cue by order interaction?	Effect found?
Single-session studies
Carpenter et al. (2009)a	Smoking, control, negative affect, neutral affect	10 min	No
Field & Duka (2004)	Smoking, control	6 min	No
Hutchison et al. (1999)b	Smoking, control	Immediately following	Yes	Cue by order interaction
Larowe et al. (2007)b	Smoking, control	10 min	No
Mahler and de Wit (2005)b,c,d	Smoking, control	10 min	Not reported
McDermut and Haaga (1998)	Smoking, control	Immediately following	Yes	Not significant e
Reid et al. (2007)c,d	Smoking, control	30 min	Not reported
Rickard-Figueroa and Zeichner (1985)	Smoking, control	Immediately following	Yes	Cue by order interaction
Sayette and Hufford (1994)	Smoking, control	Several minutes	Yes	Cue by order interaction
Shadel, Niaura, and Abrams (2001)c,d	2 smoking, control	15 min	Not reported
Shadel, Niaura, and Abrams (2004)	Smoking, control	Immediately following	No
Shadel et al. (1998)c	Smoking + negative affect, smoking + positive affect, neutral affect	Immediately following	No
Tiffany and Hakenewerth (1991)c	2 smoking, 2 control	Immediately following	No
Upadhyaya et al. (2004)c	Smoking, control	3 min	Yes	Cue by order interaction
Upadhyaya, Drobes, and Wang (2006)c,d	Smoking, control	Immediately following	Not reported
Multiple-session studies
Brody et al. (2004)c	Smoking, control	7–10 days	No
Doran et al. (2007)f	Smoking, control	Unclear-2 separate days	No
Droungas et al. (1995)	Smoking, control, negative affect	Unclear-3 separate days	No
Field et al. (2007)	Smoking, control	At least 1 week	Yes	Not significant
Franklin et al. (2007)	Smoking, control	1 hr	No
Morgan, Davies, and Willner (1999)c	Smoking, control	Unclear-2 separate days	No

Note.^a The authors report that all analyses controlled for presentation order of cues.

^bWe include these multisession studies here because the smoking and control cues were presented within a single session.

^cThe authors report presenting the cues (or cue sessions) in a “random order” or in “different orders.”

^dThe authors tested for a main effect of order (which was not found), but it is unclear whether they also tested for an interaction.

^eAn interaction was found for affect ratings but not for urge.

^fThe authors report that session order was not included in their analyses because it was found to be unrelated to dependent variables.

Interestingly, while the order effect interactions reported by Rickard-Figueroa and Zeichner and Hutchison et al. (1999) are consistent with the concern that exposure to a smoking cue may lead to a carryover effect when responding to a subsequently presented neutral cue, in some studies, the interactions involving order reflected a different pattern. Specifically, in two cases, there were greater changes in smoking cue responding when it followed a control cue (Sayette & Hufford, 1994; Upadhyaya, Drobes, & Thomas, 2004). [In this latter study, Upadhyaya et al. (2004) used just 12 adolescent smokers, and in one of the two cue presentation orders, they actually found greater urges during control cues than smoking cues. This unusual pattern distinguishes this study from most others in the literature.] This pattern is less obvious to us. Regardless of the pattern of the cue by order interaction, however, the key issue is that the presence of any cue by order interaction raises questions about the utility of counterbalancing order of cues.

We also list six studies in Table 1 that provided single exposures to smoking and control cues but did so in separate sessions. The time between sessions varied from an hour to a week or more. (When the interval is just an hour, issues related to time since last cigarette can become more troublesome [perhaps requiring subjects to smoke a cigarette prior to the second session to bring them back to a similar level of satiety], and the study may be more similar to single-session studies discussed earlier than multisession studies occurring across days.) In theory, this multisession design should be less vulnerable than single-session designs to cue by order interactions, as any carryover from the first exposure has a longer period to dissipate than in the single-session studies just described. This multisession approach is more costly to run, though, at least when it requires the same subject to show up and participate on separate days. In many cases, this means more than doubling the number of sessions needed to collect the same amount of data as in the single-session studies. (If a subject fails to show for Session 2, then Session 1 data likely are wasted.) The vast majority of these studies failed to provide any information related to potential order effects. We found just one study that provided this material. Field, Rush, Cole, and Goudie (2007) entered session order (smoking cues first vs. neutral cues first) as a between-subjects factor in their analyses and did not find any significant main effects or interactions. Notably, this study scheduled sessions at least one week apart.

In summary, most studies that counterbalance the order of presentation of smoking and control cues fail to report testing for any main effects or interactions involving order. Moreover, the single-session studies that do test for such interactions often report their presence.

Multitrial Studies

Once one chooses to present multiple trials of smoking and control cues in an experiment, then it is very likely that counterbalancing or random orders of the cues will be used. An exception is when a single “block” of smoking stimuli and a single block of control cues are presented in a fixed order. For example, Erblich and Bovbjerg (2004) presented one neutral script and one neutral in vivo exposure always before one smoking script and one smoking in vivo exposure “to avoid possible carry-over effects within exposure modality (imaginal/in vivo)” (p. 210). (Others have presented more than one type of smoking or control cue [e.g., smoking video + holding a cigarette] and considered it to be a single “combination” cue presentation [e.g., Droungas, Ehrman, Childress, & O’Brien, 1995 and Reid, Palamar, Raghavan, & Flammino, 2007, both of which are included in Table 1].) It is challenging to test for possible carryover effects in the typical multitrial experiment as the sequence of cue presentations become highly complex, and often there are not enough subjects provided with each sequence of cues to adequately test. Simply noting that across time, urges tended to increase (i.e., a significant time effect) by itself is not a problem for studies that counterbalance cue order. Rather, as noted above, the crucial test is whether the passing of time interacts with the particular type of cue being presented. This may require an elaborate set of mixed modeling analyses relying on cue type, as well as the cue type for the preceding one or two cues. Often these sets of analyses lead to separate studies in their own right (e.g., Waters et al., 2005). We located 38 smoking studies that provided a series of smoking and control cues to subjects in a random or quasi-random order. None reported any information regarding main effects or interactions with order for self-reported urge. Three of these studies reported main effects of time but did not speak to concerns about time by cue interactions. (This list of studies is available by request from the first author.) Thus, we conclude that in this popular type of research design, the possible concern that smoking cues might affect responses to subsequently delivered control cues largely has been ignored.

One blocked multitrial smoking cue study that did counterbalance order of cues and tested for order effects was conducted recently by Warthen and Tiffany (2009). These investigators developed five cue orders, with cues (smoking photos, smoking sentences, nonsmoking photos, and nonsmoking sentences) counterbalanced and randomized across participants. Two trials of the same cue reactivity type were presented over eight days with a minimum of 30 min between trials each day. For their ecological momentary assessment data, they reported a Cue Type × Order interaction for smoking cues only, in which there was significantly higher craving on the second trial than the first trial.

We located five studies that presented smoking and control cues in a fixed (unblocked) order. Four of these studies did not report testing for order effects (Dagher, Tannenbaum, Hayashi, Pruessner, & McBride, 2009; Lee, Lim, Wiederhold, & Graham, 2005; McBride, Barrett, Kelly, Aw, & Dagher, 2006; Moon & Lee, 2009). Alsene, Li, Chaverneff, and de Wit (2003) noted that there was no main effect of order but did not report testing for an order by cue interaction. Bordnick, Graap, Copp, Brooks, and Ferrer (2005) exposed subjects to a neutral and a smoking virtual reality room in a fixed order: Neutral-Smoking-Smoking-Neutral. They found that participants reported much stronger craving to the second neutral room, which followed immersion in two smoking rooms than to the first neutral room. The two smoking exposures were not different from each other, providing further evidence for cue by order interactions that are not addressed by counterbalancing.

While few multitrial studies have formally analyzed order of cue interactions, a few smoking Stroop studies have examined the effects of the initial trials relative to subsequent trials. Presumably, if carryover effects were emerging over the course of multiple presentations, the effects of cue (smoking vs. control) should dissipate over time. Waters and Feyerabend (2000) found, using a smoking Stroop study in which smoking and control words were intermingled, that the difference in response latencies produced by smoking words relative to control words did fade over time. These authors referred to the initial responses as the “acute Stroop effect” and argued that these initial responses may be less subject to carryover effects than were the later trials. Interestingly, these authors also found that the acute Stroop responses were significantly associated with time to first cigarette in the morning. Similarly, it was acute Stroop responding, rather than responses to the entire smoking Stroop task, that predicted time to relapse in a prospective study (Waters et al., 2003). These data provide indirect support for the notion that carryover effects also can affect multitrial studies and that, at least in the smoking Stroop task, the reactions to the initial cues may prove to be more meaningfully linked to outcomes of interest such as smoking relapse than the later cues that are most subject to carryover. These results also may explain the mixed findings in the literature concerning the association between cue-induced craving and cessation outcomes.

Discussion

Our review of the smoking research literature suggests that experiments presenting subjects with smoking and control cues in a counterbalanced order generally fail to consider whether order of cue presentation is affecting analyses related to key study aims. As discussed throughout this paper, if order of cues interacts with other factors in the study, then counterbalancing may fall short of comprehensively addressing order effects, and as noted by prominent research methodologists (e.g., Keppel, 1982; Winer, 1971), may even exacerbate the situation. We found that studies that counterbalanced order of smoking and control cues, or presented a series of such cues in random order, seldom mentioned testing for order effects. Indeed, several published studies failed even to provide basic information regarding how cues were presented (i.e., whether the cues were presented in a fixed, random, or quasi-random order) much less whether the order of cues was counterbalanced. When order of cues was evaluated, in many instances, order interacted with other study variables such as cue type in ways that may not be remedied by counterbalancing.

Does It Matter?

Even if counterbalancing order of cue presentation fails to prevent the occurrence of order effects, including interactions, it is important to consider just how important a problem this is for the field. If ineffective counterbalancing introduces “noise” into the study, then presumably it should become more difficult to detect significant effects. One might wonder, then, if this concern with counterbalancing is much ado about nothing. It remains unclear how best to judge the impact of inappropriately counterbalancing studies in which order interacts with key experimental variables. Many of the studies that found interactions with order also found support for their primary hypotheses. But there also are plenty of instances in which contrasts of interest were not significant (e.g., studies in which heavy and light smokers do not show differential increases in craving following smoking cue exposure), which could have been affected by order interactions that were not corrected by counterbalancing. For instance, Wilson et al. (2007) failed to observe an effect of smoking cues on a working memory task but found that this null finding was in large part a function of carryover effects. Moreover, it is unknown how many studies failing to detect significant differences went unpublished, and as noted above, some studies failing to link cue reactivity effects to subsequent relapse may have been affected by a noisy manipulation of cues due to carryover effects. When counterbalancing the order of smoking and control cues yields cue by order interactions, then power is reduced and some of the collected data (e.g., trials affected by carryover) may not be useful. Thus, selective carryover effects for certain sequences of cue presentation may lead to problems with counterbalancing and requires more consideration.

What Can Be Done?

If one decides to consider alternative approaches to counterbalancing the presentation order of smoking and control cues, there are a host of options to consider:

Counterbalance Cue Order Across Multiple Sessions

As noted above (see bottom section of Table 1), this approach has been used in quite a few studies. Although very few of them reported on order effects and thus we could not fully evaluate this approach, this design has the advantage of providing more time for carryover effects to dissipate than do single-session studies. This may be especially true when sessions are separated by at least a day or possibly a week. (It also may be true that for some research designs, intervals shorter than a day would be sufficient, and studies are needed to test the duration of carryover effects.) Clearly, future research using this approach should test for order effects, including interactions, to ensure that counterbalancing was effective. Even if sessions are days apart, there remains the possibility that exposure to smoking cues in one session might affect responding to control cues in a subsequent session. As Keppel (1982) wondered, if a prior experimental condition was disturbing or frustrating to a participant, “how will subjects react when they are told they will not be given a similar treatment in another [subsequent] condition?” (p. 378).

Insert Neutral Activities Between Cue Exposures

Some studies have attempted to minimize the possibility of carryover effects by having participants engage in another activity (e.g., watch a nature video) between smoking and control cue exposure (e.g., Carpenter et al., 2009; Colamussi, Bovbjerg, & Erblich, 2007; LaRowe, Saladin, Carpenter, & Upadhyaya, 2007). While Carpenter et al. and LaRowe et al. did not report testing for order effects, and Colamussi et al. used a fixed sequence of cues, and thus order could not be tested, it is plausible that such “cleansing” stimuli would help to reduce carryover risk. Such an intervention does, however, increase the time it takes to complete the entire cue exposure protocol, and thus, time since last cigarette could exert its own confounding effects on cravings (e.g., Sayette et al., 2001, Table 2). (We have used this approach in our smoking Stroop research [Wertz & Sayette, 2001].)

Use a Between-Subjects Design

The failsafe approach to preventing carryover effects is to conduct a between-subjects experiment, in which participants receive only smoking cues or only control cues (e.g., Juliano & Brandon, 1998; Litvin & Brandon, 2010; Warthen & Tiffany, 2009). The chief disadvantage of this approach is that it requires running far more subjects through the protocol than in a within-subjects study, which presents both types of cues to each subject. In one way, the between-subjects design resembles the multisession within-subjects design, in that each experimental session involves administering a single type of cue (i.e., smoking or control). Because participants receive just one type of cue, the between-subjects design requires recruiting more subjects than does the multisession within-subject design. The problem with attrition mentioned above for a within-subjects design is, however, no longer at issue as every completed session “counts.” Perhaps, the most fundamental point here is that if subjects can process both types of cues with minimal carryover effects, then a within-subject design permits each subject to serve as his or her own control (eliminating between-group variability), which cannot happen using this between-subject design. Of course, should that prove impossible, then researchers may need to retreat to a between-subjects approach or another alternative.

Include Order as a Factor in All Analyses

A few studies included order as a factor in their statistical models (e.g., Hutchison et al., 1999; Rickard-Figueroa & Zeichner, 1985). In theory, such an approach would control for the order of cue presentation when considering the various main effects and interactions of interest in the study. Unfortunately, many studies lack adequate power to reliably assess higher order interactions, which can lead to complex interactions that are not readily explained.

Include a “Baseline” Urge Rating Prior to Each Cue Presentation

If responding to one cue bleeds over into the response to the subsequent cue, then it may be possible to factor out this residual response by treating baseline levels as time-varying covariates in analyses (Larowe et al., 2007). Concerns with this approach, especially in multitrial procedures, include the feasibility of requiring numerous urge assessments prior to each cue presentation, which might increase assessment reactivity. Indeed, as we have argued elsewhere, the urge rating itself likely serves as a probe that cues smoking-related thinking (Sayette, Schooler, & Reichle, 2010; Sayette et al., 2000).

Provide Smoking and Control Cue in a Fixed Order

The studies reviewed above using single presentations of smoking and control cues provide evidence of cue by order interactions. Some investigators have decided that the danger of order interactions is greater than the concern with using a fixed order of cue, in which the control cue always precedes the smoking cue (e.g., Baumann & Sayette, 2006). The reasoning is that the time between the two exposures is not very long (often less than 10 or 15 min), so that the time since last cigarette is not appreciably different.

Second, exposure to a control cue often produces an urge rating that is virtually identical to “baseline” urge ratings. This point is illustrated in a study by Sayette et al. (2001). Smokers who had abstained from smoking for at least 12 hr rated their urge to smoke in a fixed order during the following four assessments: a precontrol cue baseline, exposure to a control cue, a presmoking cue baseline, and exposure to the smoking cue. Results for light smoking tobacco chippers revealed a mean urge on a 0–100 scale of 21, 23, and 25 for the first three ratings, followed by a substantial jump to 44 during the smoking cue. Similarly, heavy smokers rated their urge during the first three assessments at 49, 49, and 51, followed by 71 during the smoking cue. Urge ratings recorded during the two baseline assessments on either side of the control cue exposure suggest that exposure to a control cue is not much different than completing a simple baseline assessment.

These findings from Sayette et al. (2001) suggest that the study design was effective in revealing a powerful effect of the smoking cue relative to the control cue. [Data from Sayette et al. (2001) suggest that this same pattern holds even when subjects are permitted to smoke just before the start of the experiment, though here heavy smokers also show an effect of time since smoking even over a small amount of time.] If true, then exposing subjects first to a control cue may do little more than provide a precue baseline to subjects and would support use of a fixed order in which smoking cue follows a control cue. Such an approach also has been used in several other laboratories to address concerns with carryover effects (e.g., Adolfo, AhnAllen, & Tidey, 2009; Colamussi et al., 2007; Erblich & Bovbjerg, 2004; Traylor, Bordnick, & Carter, 2009). A limitation of this approach, though, is that it likely requires subjects to receive just one smoking cue and one control cue. As noted earlier, there is a legitimate possibility that multiple trials offer a more reliable manipulation than just a single presentation. In our view, more data are needed that contrast responses to initial “one-shot” smoking cue studies with responses to multiple trials of smoking cue (e.g., Do the patterns of reactivity from single vs. multicue studies differentially predict time to relapse?).

Consider Whether a True Cue Exposure Design Is Necessary

In some instances, it may be that the aim of a study is not to test the effect of cue-elicited responses, and alternative designs may provide useful. An example of this type of research involves examining the effects of a peak craving experience. If one wants to compare peak craving experiences in different types of smokers under controlled laboratory conditions, it may require a design that combines a smoking cue with nicotine deprivation. Sayette et al. (2005) combined smoking cues and nicotine deprivation to examine the impact of a peak craving experience on time perception (comparing responses with individuals who were not nicotine deprived and who were exposed to a control cue). In a different study, this same approach was used to examine how well smokers could predict their own future cravings while they were in either a craving or a neutral state (Sayette, Loewenstein, Griffin, & Black, 2008). This design avoids having to administer both smoking and control cues to the same subjects but is unable to disentangle the separate and interactive contributions of withdrawal and smoking cue exposure in analyzing response differences. This limitation renders the “peak-craving” induction approach inappropriate for many, though not all, research questions.

Remove the Control Cue Altogether

This unconventional research design simply uses a precue baseline assessment followed by exposure to a smoking cue without ever administering a control cue. This approach is predicated on the observation that for some measures, such as self-reported urge, exposure to a control cue seems to produce the same urge as simply completing a baseline urge assessment. The data from Sayette et al. (2001), as well as from Doran, Cook, McChargue, and Spring (2009), Rohsenow et al. (2007), and from the nonabstinent condition in Tidey, Rosenhow, Kaplan, and Swift (2005) support this point, which has led some investigators to adopt this design (e.g., Miranda, Rohsenow, Monti, Tidey, & Ray, 2008). [This issue may as yet be unresolved, though, as other studies have reported neutral cues to produce greater urges than a pre-cue baseline (Doran, Spring, & McChargue, 2007; LaRowe et al., 2007; Tidey et al., 2005 for the abstinent condition). Franklin et al. (2007) also reported a slightly higher increase in urge ratings during control cue than during a precue baseline, although in this instance the control cue assessment always occurred about 10 min after the precue baseline. In contrast, other studies have found that control cues produced lower cravings than did precue baseline assessments (Erblich & Bovbjerg, 2004; Gass, Wray, Hawk, Mahoney, & Tiffany, 2010, in preparation; Warthen & Tiffany, 2009; Wray, Godleski & Tiffany, 2010)].

If the urge rating provided by the baseline rating is similar to that produced by a control cue, then perhaps one might forgo the control cue exposure. Hutchison, LaChance, Niaura, Bryan, and Smolen (2002) note that, “Data collected in one of our previous studies suggested that control cues increased craving and other measures by only 6% of one standard deviation as compared with measures collected at baseline, which is a nonsignificant change that probably represents the effect of time … . We decided not to use control cues in the present study” (p.136). This option may be especially risky, however, if nonverbal measures of reactivity are being considered. For instance, exposure to control cues can affect cognitive measures such as secondary response time (Sayette & Hufford, 1994, Table 2) and very likely also affects psychophysiological and neurobiological responding (Franklin et al., 2007). It also shares the concern of the fixed order design in which the smoking cue always follows the baseline assessment.

Future Research

The primary conclusion we draw is that researchers should not take for granted that counterbalancing the order of smoking and control cues is appropriate. Unfortunately, it is not clear to us that there is an alternative design that is preferable. This state of affairs creates a challenge for investigators to select the best possible approach for the particular questions they wish to address and to be sensitive to its limitations (e.g., be sure to comprehensively evaluate order effects and interactions and note their implications when interpreting the study’s key findings).

Main effects and interactions involving the order of cue presentation remain an important concern for investigators, and research is indicated that evaluates these potential effects under different experimental conditions. Indeed, such research may very well require counterbalancing the order of cue presentation, but with the aim of examining factors that affect potential order effects. As noted by Tiffany (personal communication, June 2, 2010), one variant of a counterbalanced design that would explicitly investigate the details of potential carryover effects would include four conditions (between subject or across session) with two cues presented across two successive trials: (a) Smoking Cue first–Neutral Cue Second; (b) Neutral Cue first–Smoking Cue Second; (c) Smoking Cue first–Smoking Cue Second; and (d) Neutral Cue first–Neutral Cue Second. While this approach has been examined in the context of smoking Stroop studies (e.g., Waters et al., 2005), it is important to examine using more explicit smoking cues and self-reported craving measures.

A second area for future research is to continue to address mechanisms that may underlie carryover effects. Within the smoking Stroop literature, this already is beginning to happen. For instance, researchers are distinguishing between fast acting effects of cues and slow acting effects, the latter relating specifically to carryover effects. These carryover effects following smoking cues are thought to reflect a smoker’s struggle to disengage attention from emotionally salient cues (Field et al., 2009; Waters et al., 2005). Such observations suggest the possibility that individual differences in attentional disengagement may play an important etiological role in drug dependence. Thus, rather than viewing carryover effects as noise to eliminate, it instead may prove to be a topic of great scientific interest.

In summary, this review provides evidence for the existence of carryover effects produced by smoking cues and raises potential issues for studies using other drugs. Exposure to smoking-related cues can affect the processing of cues presented shortly thereafter, even when such cues are neutral. Failing to appreciate the impact of these effects may have significant implications for studies using multiple intermingled presentations of smoking-related and nonsmoking stimuli (Wilson et al., 2007). Investigating the mechanisms underlying carryover may offer insight into the nature of the cognitive biases associated with nicotine addiction.

Funding

This work was supported in part by National Institute on Drug Abuse Grant (R01 DA10605).

Declaration of Interests

None declared.