Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2026 Oct 30.
Published in final edited form as: Addict Behav. 2017 Jul 14;76:95–99. doi: 10.1016/j.addbeh.2017.07.025

Evaluating the utility of subjective effects measures for predicting product sampling, enrollment, and retention in a clinical trial of a smokeless tobacco product

Richard J O’Connor 1, Bruce R Lindgren 2, Liane M Schneller 1, Peter G Shields 3, Dorothy K Hatsukami 2
PMCID: PMC13718039  NIHMSID: NIHMS2216844  PMID: 28772248

Abstract

INTRODUCTION

Subjective effects of drugs, representing pharmacological and non-pharmacological effects, have been shown to be associated with future use and abuse. This also is the case for tobacco products and so measuring subjective effects, such as liking, satisfaction, and aversion, is crucial to gaining an understanding of consumer perception leading to increased use. This study examined the predictive validity of subjective drug and product effects with respect to product adoption.

METHODS

Smokers (N=151) were enrolled in Minneapolis, Columbus, and Buffalo. Participants were shown two snus products (Camel Snus Winterchill and Robust), asked to try each of the products for 5 minutes and to rate them using the Product Evaluation Scale (PES) and Drug Effects Questionnaire (DEQ). This was followed by a one-week use period of their preferred product and those who used at least 1 unit of Camel Snus per day (or at least 7 pouches total) were eligible to enroll in the Clinical Trial Phase assessing the impact of complete switching or dual use with smoking. Key outcomes for this study were product evaluation, extent of product use, and Clinical Trial enrollment.

RESULTS

We noted no relationships between participant characteristics such as gender, age, prior smokeless use, baseline cigarettes per day (CPD), or PES and DEQ scores with any of these outcome variables. Subjective effects were weak predictors of product use, which totaled approximately 3 units of snus per day.

CONCLUSIONS

Regardless of product, it appears that PES and DEQ ratings were uniformly poor predictors of trial enrollment and retention, though they do predict the amount of snus used during the sampling phase. Findings indicate that while subjective effects predict product preference in the short-term, they did not consistently predict extent of use or enrollment in the trial, suggesting that these initial measures have limited implications for long-term behavior.

1. Introduction

Two broad, inter-related domains of consumer perception are central to trial and adoption of new tobacco products: 1) reactions to messaging (broadly encompassing knowledge, attitudes, beliefs, and risk perceptions), and 2) responses to product use (including behavioral, sensory, and other subjective effects) (Rees et al., 2009). Subjective effects of nicotine, such as sensory factors, enjoyment, liking, and hedonics, have been shown to be associated with future use and abuse (de Wit & Phillips, 2012; Hanson, O’Connor, & Hatsukami, 2009). Measuring these sensory, affective, and cognitive elements is crucial to gaining an understanding of consumer perception of tobacco products. This is especially important in the evaluation of novel tobacco products, including those that may be marketed for reduced exposure to toxicants (e.g., modified risk tobacco products; MRTP). In allowing an MRTP claim, the Food and Drug Administration (FDA) must evaluate the generalizability of clinical trial and other findings reporting reduced exposure to toxicants and/or risk of disease from using an MRTP. Indeed, for an MRTP marketing order to be given, the manufacturer must demonstrate not only that the product reduces exposure or risk in individuals, but also that its availability would benefit the population as a whole (U.S. Department of Health and Human Services, Food and Drug Administration, & Center for Tobacco Products 2012). Part of this ‘population health’ standard entails projecting the rate of product uptake among current smokers – a product only used by a small subset of smokers, or one used only as an occasional substitute, is unlikely to have dramatic public health benefit (Institute of Medicine, 2012). Product uptake is often inferred from studies of stated preference (Biener & Bogen, 2009; Biener et al., 2016; Zhu et al., 2013), revealed preference (Rousu, O’Connor, Bansal-Travers, Pitcavage, & Thrasher, 2015; Rousu et al., 2014), simulated demand (O’Connor et al., 2014; Quisenberry, Koffarnus, Hatz, Epstein, & Bickel, 2016; Stein, Wilson, Koffarnus, Judd, & Bickel, 2017), or from short-term clinical studies (Blank & Eissenberg, 2010; Cobb, Weaver, & Eissenberg, 2010; Hatsukami et al., 2011; Hatsukami et al., 2016), each of which has strengths and limitations with respect to internal and external validity (Institute of Medicine, 2012). A key question in such short-term studies is whether subjective responses to initial exposures to novel products (e.g., a single sample in the laboratory) are reflective of potential product uptake. If initial impressions were predictive of the extent of use over a longer period, this would suggest that short-term laboratory studies could serve as important screeners for a product’s likelihood of adoption, a key consideration of public health impact. If initial ratings were poor predictors of product use, this would suggest that short-term study results could be misleading, and longer-term studies would be required to make an informed judgment.

The intention of this study was to examine the predictive validity of subjective drug and product effects with respect to product adoption, measured by use in a 2 month clinical trial. To model this, we built an initial “Sampling” phase into the clinical trial to assess whether sensory, drug effects, and experiential measures can predict who enrolls in a clinical trial of Camel Snus. The current study examines whether ratings of appeal, particularly sensory and subjective effects, are associated with degree of product use, and clinical trial enrollment and retention among current smokers.

2. Method

Data from this study come from a randomized, multi-site open-label trial examining clinical trail methods for assessing toxicity of a tobacco product (in this case snus) based on its substitution for cigarettes (NCT 01867242). The primary outcomes of the trial will be reported elsewhere. Our focus here is on the sampling phase, which was determinative of eligibility for the larger trial.

2.1. Eligibility criteria

Participants were initially eligible if they were at least 18 years of age, currently smoked at least 10 cigarettes per day (CPD), had not used smokeless tobacco for at least 3 months, were able to provide consent and read and understand study documents, and had no medical contraindications such as pregnancy, breastfeeding, uncontrolled hypertension, uncontrolled diabetes, recent myocardial infarction, or cancer. Recruitment ads targeted smokers who were interested in using oral tobacco, but not necessarily quitting: “Smokers who want to try a new oral tobacco product developed for smokers are needed for a research study that may reduce their exposure to harmful tobacco smoke.” A total of 159 individuals was enrolled in the study across 3 sites: University of Minnesota (UMN; Minneapolis, MN), Ohio State University (OSU; Columbus, OH), and Roswell Park Cancer Institute (RPCI; Buffalo, NY).

2.2. Sampling phase procedure

At an initial orientation visit, eligible participants completed a core questionnaire on smoking and tobacco use behaviors. Participants were then shown the two snus products (Camel Snus Winterchill and Camel Snus Robust), and were asked to smell then try each of the products for 5 minutes and rate the sensory effects, taste and appeal of each product immediately afterward. Participants then indicated a flavor preference (Winterchill vs. Robust), and were provided 4 tins (15 pouches per tin) of Camel Snus to use at home. Factors associated with flavor preference are reported in a separate manuscript (Schneller et al., under review). Participants were instructed to use as much or as little of the product as they wished over the next 7 days, and to record amount of daily use of both the Camel Snus and other nicotine-containing products using Interactive Voice Response (IVR). Participants were allowed to smoke during this sampling period. Participants could request and receive additional snus if more was needed. Upon return to the clinic one week later, those who used at least 1 unit of Camel Snus per day* were eligible to enroll in the Clinical Trial Phase. Participants were then randomized to one of five treatment arms differing in degree of intended substitution level for cigarettes (full vs. partial) crossed with instructions for use (specific vs. ad libitum), versus continued smoking (wait-list control).

2.3. Subjective effects

For each product, participants completed a Product Evaluation Scale (PES) (Cappelleri et al., 2007; Hatsukami, Zhang, O’Connor, & Severson, 2013), and Drug Effects Questionnaire (DEQ) (de Wit & Phillips, 2012; Morean et al., 2013) in the laboratory immediately after the five minute trial of each flavor of snus. The PES resolves to four subscales: Relief, Aversion, Psychological Reward, and Satisfaction (0–6 scales). The DEQ consists of six individual items assessing the overall desirability of the product and intentions to use in the future (0–10 scales). These items were assessed again one week later on return to the laboratory, prior to enrollment in the clinical trial phase.

Frequencies, cross-tabulations, and distributions initially characterized the data. Statistical methods included Fisher’s exact test, Chi-square test, and the non-parametric Spearman correlations and Wilcoxon rank-sum tests given non-normal distributions. Multiple regression and logistic regression models identified factors related to amount of product used and randomization status, respectively. P-values less than 0.05 were considered statistically significant. All analyses was carried out using SAS version 9.3 (SAS Institute, Inc, Cary NC).

3. Results

3.1. Participant demographics

Participants were recruited predominately from OSU (72.9%), followed by UMN (19.9%) and RPCI (7.3%). Eight participants who switched flavors during the study period were excluded from all analyses, leaving a final total of 151 participants. Males comprised 62.3% of the sample, with a mean age of 39 years (SD=12.3 years). A majority identified as White (79.5%) and non-Hispanic (97.4%). Just under half of participants had prior experience with smokeless tobacco (47.0%; though none was a current user, defined per eligibility criteria as any use in the last 3 months) and about 41% reported that their usual cigarette brand was menthol flavored. The mean baseline cigarettes per day was 17 (SD=8.0), with a median Fagerstrom Test for Nicotine Dependence (FTND) score of 3. Overall, we noted no significant relationships between participant characteristics such as sex, age, prior smokeless tobacco use, baseline CPD, and PES and DEQ scores. For the PES-psychological reward, OSU subjects had the highest mean scores and UMINN had the lowest (p=0.008); the same was true of PES-relief scores (p=0.006). For PES-aversion, OSU had the highest scores, while RPCI had the lowest (p=0.019). For DEQ-intent to use, participants recruited at RPCI had the highest score while those at UMINN had the lowest (p=0.045).

3.2. Effects of product preference and time on subjective measures

After initial trial of each flavor in the lab, 73% of participants chose Winterchill (N=110) over Robust (N=41) for their week-long sampling. For the remainder of the paper, we present scale scores corresponding to the selected product (for a more extensive analysis of flavor preferences in this study, see Schneller et al., under review). There were no significant changes across the sampling week for the DEQ scales (all p-values>0.25). Median scale scores at baseline and after one week of sampling are shown in Table 1. There was a small but significant decrease in satisfaction after one week of sampling the preferred flavor (median decrease=0.25 units, p=0.023). We observed small but significant increases in the psychological reward (median increase=0.4 units, p<0.001), aversion (median increase=0.25, p<0.001) and relief subscales (median increase=0.4 units, p=0.018) of the PES after one week of sampling.

Table 1:

Correlating subjective measures (PES, DEQ) at the time of sampling (baseline) of the selected flavor and after one week of sampling (follow up) this flavor (N=151).

PES Subscale Baseline Follow up N Corr* (p-value)
N Median (min/max) N Median (min/max)
Satisfaction 151 3.0 (0.0/6.0) 134 3.0 (0.0/6.0) 134 0.50 (<0.001)
Psychological 151 1.2 (0.0/6.0) 134 2.1 (0.0/5.2) 134 0.59 (<0.001)
Aversion 151 0.25 (0.0/4.5) 134 0.75 (0.0/3.75) 134 0.38 (<0.001)
Relief 151 2.6 (0.0/6.0) 134 3.0 (0.0/6.0) 134 0.56 (<0.001)
DEQ Questions Baseline Follow up N Corr* (p-value)
N Median (min/max) N Median (min/max)
Feel any effects? 138 2.0 (0.0/10.0) 126 3.0 (0.0/10.0) 118 0.40 (<0.001)
Any good effects? 145 4.0 (0.0/10.0) 123 4.0 (0.0/10.0) 121 0.42 (<0.001)
Any bad effects? 129 0.0 (0.0/8.0) 118 1.0 (0.0/10.0) 108 0.30 (0.002)
Like the product? 150 5.0 (0.0/10.0) 131 5.0 (0.0/10.0) 131 0.52 (<0.001)
Desire the product? 147 3.0 (0.0/10.0) 127 3.0 (0.0/10.0) 125 0.56 (<0.001)
Use the product again? 151 5.0 (0.0/10.0) 128 4.5 (0.0/10.0) 128 0.62 (<0.001)
*

Spearman Correlation coefficient.

3.3. Product use during sampling

The number of days in the sampling period ranged from 4 to 14 with 77.4% of the participants reporting 7 days. The snus per day ranged from 0.11 to 12.14 with a median of 3.00 for the 137 participants with complete data. There was no significant difference between Winterchill and Robust in amount of snus used per day during sampling (3.2 vs 2.8 units per day, p=0.925). Subjective effects were weak predictors of the extent of product use. PES-satisfaction (r=0.29, p=0.001) and DEQ-liking (r=0.24, p=0.006) were positively associated with snus per day, while PES-aversion (r=−0.21, p=0.012) and DEQ-bad effect (r=−0.29, p=0.001) were negatively associated with snus per day. Table 2 summarizes multivariate analyses of the relationship between scale scores and the reported number of snus used per day during the sampling week. Model 1 includes product flavor and the PES scale scores; Model 2 flavor and DEQ item scores. Model 3 included the strongest effects from Models 1 and 2 (p < 0.10), adjusted for flavor. PEQ-satisfaction showed a strong positive relationship with snus per day (i.e., greater satisfaction was associated with greater use), while DEQ-bad effects showed a modest negative relationship (i.e, more reported bad effects was associated with less use).

Table 2.

Multiple Regression for average snus per day during sampling week

Predictor b 95% CI P-value
Model 1 (n=137) R-squared=0.095
Robust (vs Winterchill) 0.24 −0.56, 1.04 0.556
PES-Satisfaction (1 unit ↑) 0.40 0.05, 0.75 0.025
PES-Psychological (1 unit ↑) 0.16 −0,19, 0.51 0.364
PES-Aversion (1 unit ↑) −0.19 −0.66, 0.27 0.414
PES-Relief (1 unit ↑) −0.15 −0.54, 0.24 0.456
Model 2 (n=115) R-squared=0.102
Robust (vs Winterchill) 0.12 −0.71, 0.95 0.774
DEQ-Feel any effects? (1 unit ↑) 0.01 −0.17, 0.19 0.905
DEQ-Any good effects? (1 unit ↑) −0.04 −0.25, 0.17 0.691
DEQ-Any bad effects? (1 unit ↑) −0.21 −0.43, 0.02 0.068
DEQ-Like the product? (1 unit ↑) 0.16 −0.07, 0.38 0.166
DEQ-Desire the product? (1 unit ↑) 0.01 −0.23, 0.25 0.936
DEQ-Use the product again? (1 unit ↑) 0.003 −0.23, 0.24 0.980
Model 3 (n=117) R-squared=0.127
Robust (vs Winterchill) 0.24 −0.53, 1.02 0.538
PES-Satisfaction (1 unit ↑) 0.40 0.13, 0.68 0.004
DEQ-Any bad effects? (1 unit ↑) −0.17 −0.37, 0.03 0.088

We also examined whether changes in PES or DEQ scores over the sampling week were related to product use. Changes in “Desire the product” were positively correlated with product use (r=0.21, p=0.020, n=125). Product use was also correlated with change in “use the product again” scores (r=0.25, p=0.004, n=128) and PES-satisfaction (r=0.19, p=0.031, n=134).

3.4. Predictors of trial enrollment and retention

Of the initial 151 participants, 47% (N=71) met use criteria to be randomized (i.e., >1 snus units per day during sampling phase or 7 total) and desired to continue in the clinical trial phase of the study. Those who were randomized were significantly older on average (42.0 y vs 34.5 y, p<.001), and were marginally likely to have lower FTND scores (median 3 vs 4, p = 0.098), but showed no differences by site, gender, race/ethnicity, prior smokeless tobacco use, menthol cigarette preference, or baseline CPD (p’s >0.17; see Supplemental Table 1). Table 3 shows median scores on the PES and DEQ scales by randomization status -- no significant differences in scores were observed between the groups. Multivariate logistic regression models adjusting for flavor and number of snus per day showed than PES scale and DEQ item scores were not significantly associated with randomization status (see Supplemental Table 2).

Table 3.

Product evaluation scale (PES) and Drug Effects Questionnaire (DEQ) by randomization status and trial retention.

PES Subscale Not Randomized Randomized P-value*
N Median (min/max) N Median (min/max)
Satisfaction 80 3.0 (0.0/6.0) 71 3.0 (0.0/5.8) 0.297
Psychological 80 1.2 (0.0/6.0) 71 1.4 (0.0/5.0) 0.348
Aversion 80 0.1 (0.0/4.5) 71 0.3 (0.0/3.8) 0.822
Relief 80 2.6 (0.0/6.0) 71 2.6 (0.6/6.0) 0.224
DEQ Item Not Randomized Randomized P-value*
N Median (min/max) N Median (min/max)
Feel any effects? 73 2.0 (0.0/10.0) 65 3.0 (0.0/10.0) 0.338
Any good effects? 75 4.0 (0.0/10.0) 70 3.0 (0.0/10.0) 0.732
Any bad effects? 69 0.0 (0.0/6.0) 60 0.0 (0.0/8.0) 0.741
Like the product? 79 5.0 (0.0/10.0) 71 5.0 (0.0/10.0) 0.510
Desire the product? 78 3.5 (0.0/10.0) 69 3.0 (0.0/9.0) 0.964
Use product again? 80 5.0 (0.0/10.0) 71 5.0 (0.0/10.0) 0.684
Subscale In study ≤ week 2 In study > week 2 P-value*
N Median (min/max) N Median (min/max)
Satisfaction 21 3.5 (0.0/5.5) 50 3.0 (0.8/5.8) 0.574
Psychological 21 1.4 (0.2/5.0) 50 1.6 (0.0/5.0) 0.714
Aversion 21 0.3 (0.0/3.8) 50 0.1 (0.0/3.0) 0.180
Relief 21 3.0 (0.6/6.0) 50 2.6 (1.0/6.0) 0.289
Questions In study ≤ week 2 In study > week 2 P-value*
N Median (min/max) N Median (min/max)
Feel any effects? 21 2.0 (0.0/10.0) 44 3.0 (0.0/9.0) 0.949
Any good effects? 21 4.0 (0.0/10.0) 49 3.0 (0.0/10.0) 0.377
Any bad effects? 19 0.0 (0.0/7.0) 41 0.0 (0.0/8.0) 0.642
Like the product? 21 5.0 (1.0/10.0) 50 5.0 (0.0/10.0) 0.709
Desire the product? 21 3.0 (0.0/8.0) 48 4.0 (0.0/9.0) 0.470
Use product again? 21 5.0 (0.0/10.0) 50 5.0 (0.0/10.0) 0.780
*

The p-value is based on the Wilcoxon rank sum test.

Of the 71 individuals randomized, 41 (57.8%) completed all 8 weeks of the trial. Of those who dropped, 21 did so by the second week of the trial. However, none of the PES scales or DEQ items differed significantly (p’s >.180; see Table 3) between those who dropped in the first 2 weeks versus those who persisted longer. Overall, we noted a negative correlation between PES-aversion and week of dropout (rs= −0.25, p = .035), suggesting that higher aversion was associated with earlier dropout.

4. Discussion

The current study sought to examine whether initial ratings of a novel product (Camel Snus) could predict the extent of use of a preferred product (a proxy measure of uptake), and enrollment and retention in a clinical trial involving that product (a proxy measure of continued use). Findings indicate that while subjective effects modestly predicted uptake of the product, and did not consistently predict continued use. Flavor preference (Wintergreen vs. Robust) was also not associated with amount used, nor continued use, though was associated with subjective effects (see Schneller et al., under review). This suggests that initial measures may permit only limited inferences about longer-term behavior.

Ratings on the PES and DEQ scales were generally low (e.g., median of 5 on a 10 point scale for liking, and median 3 on a 10 point scale on desire), indicating that Camel Snus was not particularly appealing to the participants. These findings are broadly consistent with prior studies that have shown low acceptability of snus amongst smokers, ranging from laboratory studies (Blank & Eissenberg, 2010; Cobb et al., 2010) to population uptake data (Biener, McCausland, Curry, & Cullen, 2011; Biener et al., 2016). The level of Camel Snus use per day was also relatively low, with a median of 3 units per day, suggesting that snus was not a robust substitute for cigarette smoking. That said, PES-satisfaction and DEQ-liking scores on the initial trial did predict extent of use, and this is sensible – one would expect greater use of a satisfying product that the participant likes. This is consistent with an earlier study suggesting that PES-satisfaction scores predicted extent of use in a clinical trial context (Hatsukami et al., 2013); (Hatsukami et al., 2016).

Just under half of participants who entered the sampling phase went on to enroll in the clinical trial portion, and of these about 30% dropped out within 2 weeks. However, it does not appear that the product ratings (PES or DEQ) differed substantially between those who did and did not continue. Given the commonality of low scores, it is possible that most participants disliked the product, and the primary difference between those who did and did not enroll was in their willingness to use a less desirable product (relative to cigarettes) for the duration of the trial. This suggests that including this phase may have served a purpose by removing prospective participants who might not comply with directions or use enough of the product to make proper conclusions about the effects of that product.

This study is subject to a number of limitations. Participants were enrolled at multiple sites, and while a common protocol was used, small differences in the implementation of the sampling protocols or environmental characteristics could bias findings toward the null. We also identified differences in regional responses to (though not necessarily use of) a product. Therefore, it is important to take this into consideration when testing products and conducting trials – generalizability of single site studies maybe be limited. The current study has a somewhat narrow focus on whether the initial ratings serve as useful indicators of trial enrollment and retention (including DEQ, which was not used in the prior Hatsukami et al, 2013 study). The prior study also offered a wide array of products (Hatsukami et al, 2013), while this one was focused on a particular product – similar to what a manufacturer might be required to conduct for a trial supporting a modified risk claim. Because this was a prelude to a clinical trial, many potential participants were excluded for health reasons, and so our findings may have limited generalizability. Finally, because the test product was not particularly liked by subjects, the pattern of results may not generalize to other potential MRTPs, meaning such studies should be repeated with e-cigarettes or other novel devices.

The intention of this study was to examine the predictive validity of subjective drug and product effects with respect to product adoption. The current study found that subjective effects predict product use in the short-term but not enrollment in the trial. This suggests that initial responses to a product might be associated with the extent of use in the short-term, but not necessarily continued use.

Supplementary Material

1

Role of funding sources

This work was supported by the National Cancer Institute U19CA157345.

Footnotes

Conflict of interest

All authors declare that they have no conflicts of interest.

*

Initially the criterion was at least 14 pouches total, but was later reduced to at least 7 pouches total because so few people met the initial criterion.

References

  1. Biener L, & Bogen K. (2009). Receptivity to Taboka and Camel Snus in a U.S. test market. Nicotine Tob Res, 11(10), 1154–1159. doi: 10.1093/ntr/ntp113 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Biener L, McCausland K, Curry L, & Cullen J. (2011). Prevalence of trial of snus products among adult smokers. Am J Public Health, 101(10), 1874–1876. doi: 10.2105/AJPH.2010.200097 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Biener L, Roman AM, Mc Inerney SA, Bolcic-Jankovic D, Hatsukami DK, Loukas A, … Romito L. (2016). Snus use and rejection in the USA. Tob Control, 25(4), 386–392. doi: 10.1136/tobaccocontrol-2013-051342 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Blank MD, & Eissenberg T. (2010). Evaluating oral noncombustible potential-reduced exposure products for smokers. Nicotine Tob Res, 12(4), 336–343. doi: 10.1093/ntr/ntq003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Cappelleri JC, Bushmakin AG, Baker CL, Merikle E, Olufade AO, & D.G. G. (2007). Confirmatory factor analyses and reliability of the modified cigarette evaluation questionnaire. Addict Behav, 32(5), 912–923. [DOI] [PubMed] [Google Scholar]
  6. Cobb CO, Weaver MF, & Eissenberg T. (2010). Evaluating the acute effects of oral, non-combustible potential reduced exposure products marketed to smokers. Tob Control, 19(5), 367–373. doi: 10.1136/tc.2008.028993 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. de Wit H, & Phillips TJ. (2012). Do initial responses to drugs predict future use or abuse? Neurosci Biobehav Rev, 36(6), 1565–1576. doi: 10.1016/j.neubiorev.2012.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Hanson K, O’Connor R, & Hatsukami D. (2009). Measures for assessing subjective effects of potential reduced-exposure products. Cancer Epidemiol Biomarkers Prev, 18(12), 3209–3224. doi: 10.1158/1055-9965.EPI-09-0971 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Hatsukami DK, Jensen J, Anderson A, Broadbent B, Allen S, Zhang Y, & Severson H. (2011). Oral tobacco products: preference and effects among smokers. Drug Alcohol Depend, 118(2–3), 230–236. doi: 10.1016/j.drugalcdep.2011.03.026 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Hatsukami DK, Severson H, Anderson A, Vogel RI, Jensen J, Broadbent B, … Hecht SS. (2016). Randomised clinical trial of snus versus medicinal nicotine among smokers interested in product switching. Tob Control, 25(3), 267–274. doi: 10.1136/tobaccocontrol-2014-052080 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Hatsukami DK, Zhang Y, O’Connor RJ, & Severson HH. (2013). Subjective responses to oral tobacco products: scale validation. Nicotine Tob Res, 15(7), 1259–1264. doi: 10.1093/ntr/nts265 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hatsukami DK, Vogel RI, Severson HH, Jensen JA, O’Connor RJ. (2016). Perceived Health Risks of Snus and Medicinal Nicotine Products. Nicotine Tob Res, 18(5), 794–800. doi: 10.1093/ntr/ntv200 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Institute of Medicine. (2012). Scientific Standards for Studies on Modified Risk Tobacco Products. Washington, DC: The National Academy of Sciences. [Google Scholar]
  14. Morean ME, de Wit H, King AC, Sofuoglu M, Rueger SY, & O’Malley SS. (2013). The drug effects questionnaire: psychometric support across three drug types. Psychopharmacology (Berl), 227(1), 177–192. doi: 10.1007/s00213-012-2954-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. O’Connor RJ, June KM, Bansal-Travers M, Rousu MC, Thrasher JF, Hyland A, & Cummings KM. (2014). Estimating demand for alternatives to cigarettes with online purchase tasks. Am J Health Behav, 38(1), 103–113. doi: 10.5993/AJHB.38.1.11 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Quisenberry AJ, Koffarnus MN, Hatz LE, Epstein LH, & Bickel WK. (2016). The Experimental Tobacco Marketplace I: Substitutability as a Function of the Price of Conventional Cigarettes. Nicotine Tob Res, 18(7), 1642–1648. doi: 10.1093/ntr/ntv230 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Rees VW, Kreslake JM, Cummings KM, O’Connor RJ, Hatsukami DK, Parascandola M, … Connolly GN. (2009). Assessing consumer responses to potential reduced-exposure tobacco products: a review of tobacco industry and independent research methods. Cancer Epidemiol Biomarkers Prev, 18(12), 3225–3240. doi: 10.1158/1055-9965.EPI-09-0946 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Rousu MC, O’Connor RJ, Bansal-Travers M, Pitcavage JM, & Thrasher JF. (2015). The impact of free trial acceptance on demand for alternative nicotine products: evidence from experimental auctions. Harm Reduct J, 12, 18. doi: 10.1186/s12954-015-0052-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Rousu MC, O’Connor RJ, Thrasher JF, June KM, Bansal-Travers M, & Pitcavage J. (2014). The impact of product information and trials on demand for smokeless tobacco and cigarettes: evidence from experimental auctions. Prev Med, 60, 3–9. doi: 10.1016/j.ypmed.2013.11.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Stein JS, Wilson AG, Koffarnus MN, Judd MC, & Bickel WK. (2017). Naturalistic assessment of demand for cigarettes, snus, and nicotine gum. Psychopharmacology (Berl), 234(2), 245–254. doi: 10.1007/s00213-016-4455-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. U.S. Department of Health and Human Services, Food and Drug Administration, & Center for Tobacco Products (2012). Modified Risk Tobacco Product Applications. Retrieved from Rockville, MD: https://www.fda.gov/downloads/TobaccoProducts/GuidanceComplianceRegulatoryInformation/UCM297751.pdf [Google Scholar]
  22. Zhu SH, Gamst A, Lee M, Cummins S, Yin L, & Zoref L. (2013). The use and perception of electronic cigarettes and snus among the U.S. population. PLoS One, 8(10), e79332. doi: 10.1371/journal.pone.0079332 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES