Subjective Responses to Oral Tobacco Products: Scale Validation

Abstract

Introduction:

Several noncombusted oral tobacco products have been introduced that are primarily marketed to cigarette smokers. An important component of evaluating these products involves assessment of subjective responses to the product. To date, few studies have been undertaken to examine the validity of subjective response questionnaires for oral tobacco products. The goal of this study is to examine the extent subjective responses to a product are related to product preference and extent of product use.

Methods:

Data from a study examining oral tobacco product preference were used. Smokers were asked to sample a variety of oral tobacco products that differed in formulation (snus versus dissolvables) and dose of nicotine. At the end of the sampling period, subjects were asked to choose the product that they would use to completely substitute for cigarettes for the next 2 weeks. During the sampling period, subjects completed a Product Evaluation Scale (PES) that describes subjective responses to the product. During the treatment phase, they kept record of amount of product use.

Results:

Subjective responses to the product on the PES were related to product choice and to some extent, the amount of product use. Product choice was associated with different characteristics of the product and smoker needs.

Conclusion:

The PES may be a useful tool for the evaluation or oral tobacco products.

INTRODUCTION

The Family Smoking Protection and Tobacco Control Act requires that a new tobacco product, product standard, or modified risk tobacco product be evaluated based on its impact on public health. Public health impact is determined by the toxicity of the product, extent of uptake of the product, persistent use of the product, and the concurrent use with other tobacco products (Institute of Medicine, 2012). A conceptual model of tobacco product use holds that consumer response to a product, comprised both of product beliefs and subjective evaluations, relates to product uptake (Rees et al., 2009). Furthermore, a significant part of tobacco product evaluation involves determining its abuse liability (or potential for persistent use), which can include an individual’s subjective response to the product (Carter et al., 2009). Although several types of scales have been used to assess different drugs on their use or abuse potential (e.g., Carter & Griffiths, 2009), few validated scales have been developed for tobacco products (Carter et al., 2009; Institute of Medicine, 2012; Hanson, O’Connor, & Hatsukami, 2009).

Prior studies have used different methods to make these product assessments (Blank & Eissenberg, 2010; Cobb, Weaver, & Eissenberg, 2010; Gray, Breland, Weaver, & Eissenberg, 2008; Kotlyar et al., 2011; Mendoza-Baumgart et al., 2007; O’Connor et al., 2011). Yet, no studies have systematically and specifically examined how subjective responses to products are linked to actual uptake or use of the product. This area of investigation is important. In recent years, several novel oral tobacco products have entered the U.S. tobacco market. These products include spitless smokeless tobacco or snus contained in small packets and dissolvable tobacco products. These products are being marketed to the smoker as substitutes for smoking and/or for use in situations where smoking is prohibited. Little is known about the palatability or extent of potential uptake of these oral tobacco products among smokers.

The purpose of this study is primarily methodological. We sought to determine if subjective responses to oral tobacco products, using the Product Evaluation Scale, are related to product preference and extent of product use, using the data collected from our prior study (Hatsukami et al., 2011). This analysis will be valuable in validating a tool to help estimate potential for uptake and continued use of a product. The results will provide further direction in the types of scales that could be used or further developed.

SUBJECT AND METHODS

Details of the study design and overall findings are discussed elsewhere (Hatsukami et al., 2011). Briefly, smokers who were interested in stopping smoking were recruited at two sites (Minneapolis/St Paul, MN and Eugene, OR) into a study that was described as exploring a tobacco product that was alternative to smoking. Subjects underwent a sampling period, which involved sampling five different products that varied by formulation (snus versus dissolvables) and free nicotine content. Per portion, General Snus had 3.37mg of free nicotine, Camel Snus had 1.74–1.97mg, Stonewall had 0.28–0.57mg, Marlboro Snus had 0.14–0.38mg, and Ariva had 0.24–0.25mg (Dr. Irina Stepanov, personal communication). Subjects were blind to the brand names of snus to limit brand extension effects. During the sampling period, subjects were assigned to a specific, randomly determined order for trying the products, and sampled only one product on separate days during a 2-week sampling period.

During the sampling day, subjects were required to abstain from smoking from waking time until 1 p.m. or for least a 5-hr period of time. They were asked to sample at least three portions of the product assigned for that day. Subjects were asked to rate the product at 30min after the third portion using the Product Evaluation Scale (PES). After this time, they were free to resume smoking or continue to use the product. This sampling day was followed by the resumption of smoking cigarettes for 1 day before the next product was sampled. At the end of this sampling period, subjects were asked to choose the product that they would like to use during the 2-week smoking abstinence period. During the 2-week abstinence period, the subjects recorded the number of products and cigarettes used per day on a daily basis and subjective responses to the oral tobacco product using the PES on Days 2, 7, and 14.

Measures

The PES adapted items from the modified Cigarette Evaluation Questionnaire (mCEQ, Cappelleri et al., 2007; Westman, Levin, & Rose, 1992) to conform to rating oral tobacco products. The PES included items from the mCEQ subscales for satisfaction, psychological reward, and aversion. Additional items such as sensation in the mouth (in the case of oral tobacco products), questions on reduction of craving and withdrawal, and items that might be associated with the use of specific oral products were also included. Subjects rated responses on a 7-point Likert type scale (1 was described as not at all and 7 as extremely). Table 1 shows the items for this scale.

Table 1.

Items From the Product Evaluation Scale

Product Evaluation Scalea
1. Was it satisfying?
2. Did it taste good?
3. Did you enjoy the sensations in your mouth?
4. Did it calm you down?
5. Did it make you feel more awake?
6. Did it make you feel less irritable?
7. Did it help you concentrate?
8. Did it reduce your hunger for food?
9. Did it make you dizzy?
10. Did it make you nauseous?
11. Did it immediately relieve your craving for a cigarette?
12. Did you enjoy it?
13. Did it relieve withdrawal symptoms?
14. Did it relieve the urge to smoke?
15. Was it enough nicotine?
16. Was it too much nicotine?
17. Was it easy to use?
18. Were there bothersome side effects?
19. Were you comfortable using the product in public?
20. Did you still have a craving for a cigarette after using the product?
21. Are you concerned that you would become dependent on this product?

Notes. ^aRated on a 1–7 scale, where 1 (not at all) and 7 (extremely); subscale scores were the following: satisfaction (items 1, 2, 3, and 12); psychological reward (items 4, 5, 6, 7, and 8); aversion (items 9, 10, 16, and 18); relief (items 11, 13, 14, 15, and reversed for 20), item scores for each subscale were averaged.

Analysis Plan

The goals of the analyses were to determine: (a) underlying factor structure of the PES, (b) subject responses on the PES across different oral tobacco products, (c) the relationship between responses on the PES during sampling and product choice after sampling, and (d) the association between PES and amount of product use, both assessed during the cigarette cessation period. Statistics used to analyze these goals are described in the next section along with corresponding results.

RESULTS

Subjects

Ninety-nine subjects entered the sampling phase (N = 55 in Minnesota and N = 44 in Oregon) and 97 entered the cessation phase. Among those who entered the sampling period, the mean age was 40.1 (SD = 13.2), 64 subjects were male, 64 had greater than a high school education, 91 subjects were employed. With regards to tobacco use history, the mean cigarettes smoked per day was 19.8±8.1, duration of smoking was 15.7±12.4 years, Fagerström Test for Nicotine Dependence (FTND) score was 5.1±2.1, and the mean motivation to quit was 9.14±1.01 on a 10-point scale.

Factor Structure for the PES

Factor analysis was conducted for the PES data using the principal factor method for each sampled product. The purpose was to explore the possible underlying factor structure of the set of 21 PES items and simplify the interrelated measures before testing the scale. In each principal factor analysis model, squared multiple correlations that indicate amount of variance explained by each common factor were calculated; factors were rotated using orthogonal varimax method. Items with larger than 0.55 rotated factor loadings were considered to be highly correlated to a common factor. Through this analysis, the 21 PES items were summarized into four common factors (satisfaction, psychological reward, aversion, and relief; see Table 1 for items associated with each factor) and three individual items (item 17 “easy to use,” item 19 “comfortable using the product in public,” and item 21 “concerned about dependence on the product”). The general grouping of these items were observed across at least three of the oral tobacco products.

Subjective Responses During Sampling Period

The primary goal of this analysis was to examine how subjects rated each of the products and differences in responses to these products. Because during the sampling period each participant rated five products for each PES subscale or item and the responses from the same participant were correlated, general linear mixed models were used. Specifically, subjective responses to the four PES subscales and three PES items collected during the sampling period were analyzed using seven general linear mixed models, one for each PES subscale or item. Each model included random intercepts as a random effect and the following categorical covariates as the fixed effects: (a) site, (b) sampling day, (c) product sampled on a sampling day, (d) prior product sampled, (e) interaction between product sampled and prior product sampled, and (f) interaction between product sampled and sampling day. Sampling day and prior product sampled were to reflect potential sampling order effect and carry-over effect, respectively. The final model for each outcome was determined using backward selection. P values of multiple comparisons for significant fixed effects were adjusted by Tukey’s method.

Our analysis shows that product was the only significant factor for all of the PES responses (Table 2, all p values <.01) except aversion for which both products (Table 2, p < .0001) and prior product (p = .018) were significant. The following are the results of multiple comparisons for the significant product effect. Compared with any of the other four products, smokers rated General Snus as producing significantly less satisfaction (p < .0001 for all), more aversion (p < .0001 for all), and less easy to use (p = .003 to p < .0001). Compared with Camel Snus, General Snus was also rated as resulting in significantly less psychological reward (p = .002), less relief (p = .017), and less concern about dependence (p = .003). Marlboro Snus and Camel Snus both were significantly less comfortable for use in public than Stonewall (p = .002 and p = .010, respectively). This was also the case when Marlboro Snus was compared with Ariva (p = .030). General Snus was significantly less comfortable for use in public than any of the other four products (p = .004 to p < .0001). In addition, prior product effect on aversion shows that smokers felt more averseness of a product when the prior product they sampled was Ariva or Camel Snus than when it was General Snus (p = .045 and p = .053, respectively). Specifically, when the prior product was Ariva or Camels Snus the least squares mean aversion scores of subsequently sampled product were 2.13 and 2.12, respectively. However, when prior product was General Snus, the least squares mean aversion of subsequent product sampled was low (1.70).

Table 2.

Response to Product on the Product Evaluation Scale During Sampling Period (N = 98^a)

Variableb	Subjective response score when sampling products (least square mean [SE])
	General Snus	Camel Snus	Marlboro Snus	Stonewall	Ariva	Product effect p value
Satisfaction	1.75 (0.17)	3.64 (0.17)	3.59 (0.16)	3.32 (0.17)	3.41 (0.17)	p < .0001
Psychological reward	2.20 (0.11)	2.57 (0.11)	2.46 (0.11)	2.42 (0.11)	2.43 (0.11)	p = .0048
Aversion	2.73 (0.11)	1.93 (0.11)	1.70 (0.11)	1.86 (0.11)	1.64 (0.11)	p < .0001
Relief	3.35 (0.13)	3.80 (0.13)	3.71 (0.13)	3.60 (0.13)	3.45 (0.13)	p = .0123
Easy to use	4.79 (0.17)	5.72 (0.17)	5.50 (0.17)	5.78 (0.17)	5.89 (0.17)	p < .0001
Comfortable using	3.90 (0.21)	4.87 (0.20)	4.75 (0.20)	5.64 (0.20)	5.44 (0.20)	p < .0001
Concerned about dependence	1.51 (0.14)	2.02 (0.14)	1.87 (0.14)	1.77 (0.14)	1.79 (0.14)	p = .0090

Notes. ^aOne subject dropped during the sampling phase.

^bRange of scores is from 1 to 7 with 1 described as not at all and 7 as extremely.

Relationship Between Subjective Responses During Sampling and Product Choice After Sampling

We evaluated the validity of PES by examining the relationship between PES during sampling and choice of product after sampling. In this analysis, PES subscale and item scores were outcome variables and because they were repeated measures, general linear mixed models were used. Each model included sampled product, product choice, and their interaction as the categorical fixed effects, and random intercepts as a random effect. The assumption of no carry-over effect had been confirmed previously for all the PES items except aversion. Thus, prior product was included in the mixed model for aversion as an additional fixed effect. In order to understand the relationship between subjective responses during sampling and choice of product after sampling, in each model, we first determined whether the subjects PES subscale or item score when sampling this product would be significantly better than the average of the scores when the other four products were sampled. This was conducted using the “estimate” statement of the “mixed” procedure in SAS. We also examined the differences in PES scores across the five sampled products within each choice group (using the “slice” option of “lsmeans” statement in the “mixed” procedure) and also compared the chosen product with each of the four nonchosen products for each subscale and item (using the “lsmestimate” statement in the “mixed” procedure). These analyses showed patterns of results (data not shown) similar to the first analysis. We present results of the first analysis to simplify discussion of results.

As shown in Table 3, there was a significant relationship between subjective responses during the time of sampling and product choice for some of items and depended on product characteristics for other items. In the product choice phase, no one chose General Snus, which was uniformly rated unfavorably during the product sampling phase. During sampling, the chosen product typically had higher mean scores compared with the average of other four sampled products for items related to satisfaction (p = .001 to p < .0001) and relief (p = .057 when choice was Ariva and p = .002 to p < .0001 for other product choices), and generally lowest mean score for aversion (p = .048 to p = .001). For subjects who chose the products that had higher levels of nicotine (Camel Snus and Stonewall), they rated them significantly higher than the average of other four products on psychological reward (p < .0001 and p = .004, respectively) and concerned about dependence (p = .003 and p = .009, respectively) when sampling those products. For subjects who chose the dissolvable products (Ariva and Stonewall), their mean scores for ease of use (p = .006 and p = .049, respectively) and comfortable using in public (p = .003 and p = .004, respectively) were significantly higher than the average of the mean scores of the other products during the sampling period.

Table 3.

Response to Product on the Product Evaluation Scale During Sampling Period and Choice of Product at the End of Sampling Period

Subjective response	Choice of product at the end of sampling period (N)	Subjective response score when sampling products (least square mean [SE])					Hypothesis testinga
		General Snus	Camel Snus	Marlboro Snus	Stonewall	Ariva	p value
Satisfaction	Camel Snus (N = 27)	1.98 (0.29)	4.75 (0.28)	4.09 (0.29)	3.30 (0.30)	3.68 (0.30)	<.0001
	Marlboro Snus (N = 23)	1.82 (0.32)	3.85 (0.33)	4.09 (0.32)	2.85 (0.31)	2.88 (0.31)	<.0001
	Stonewall (N = 24)	1.68 (0.29)	2.82 (0.30)	3.53 (0.29)	3.96 (0.29)	3.59 (0.30)	<.0001
	Ariva (N = 24)	1.42 (0.34)	2.84 (0.36)	2.44 (0.33)	2.95 (0.36)	3.43 (0.34)	.0010
Psychological reward	Camel Snus (N = 27)	2.51 (0.20)	3.21(0.20)	2.75 (0.20)	2.53 (0.20)	2.53 (0.20)	<.0001
	Marlboro Snus (N = 23)	2.17 (0.22)	2.47 (0.21)	2.53 (0.21)	2.10 (0.21)	2.33 (0.21)	.0934
	Stonewall (N = 24)	2.28 (0.21)	2.47 (0.21)	2.67 (0.21)	2.96 (0.21)	2.66(0.22)	.0043
	Ariva (N = 24)	1.82 (0.21)	2.08 (0.21)	1.90 (0.21)	2.12 (0.21)	2.20 (0.021)	.1470
Aversion	Camel Snus (N = 27)	2.38 (0.21)	1.44 (0.20)	1.49 (0.20)	1.77 (0.21)	1.79 (0.21)	.0231
	Marlboro Snus (N = 23)	2.33 (0.22)	1.97 (0.22)	1.57 (0.22)	2.02 (0.22)	1.54 (0.22)	.0482
	Stonewall (N = 24)	3.04 (0.22)	2.42 (0.22)	1.86 (0.22)	1.77 (0.21)	1.66 (0.22)	.0155
	Ariva (N = 24)	3.19 (0.22)	1.97 (0.21)	1.92 (0.21)	1.87 (0.22)	1.60 (0.21)	.0012
Relief	Camel Snus (N = 27)	3.58 (0.25)	4.56 (0.24)	3.88 (0.24)	3.44 (0.25)	3.35 (0.25)	<.0001
	Marlboro Snus (N = 23)	3.36 (0.27)	3.66 (0.26)	3.98 (0.26)	3.10 (0.26)	3.10 (0.26)	.0024
	Stonewall (N = 24)	3.34 (0.26)	3.27 (0.26)	3.71 (0.26)	4.33 (0.26)	3.53 (0.26)	<.0001
	Ariva (N = 24)	3.12 (0.26)	3.64 (0.26)	3.26 (0.26)	3.58 (0.26)	3.82 (0.26)	.0572
Easy to use	Camel Snus (N = 27)	5.30 (0.33)	6.04 (0.33)	5.93 (0.33)	5.48 (0.33)	5.78 (0.33)	.1552
	Marlboro Snus (N = 23)	4.80 (0.36)	5.48 (0.35)	5.30 (0.35)	5.48 (0.35)	5.57 (0.35)	.9368
	Stonewall (N = 24)	5.06 (0.35)	6.04 (0.35)	5.63 (0.35)	6.38 (0.35)	6.33 (0.35)	.0494
	Ariva (N = 24)	3.96 (0.35)	5.25 (0.35)	5.08 (0.35)	5.82 (0.35)	5.88 (0.35)	.0063
Comfortable using	Camel Snus (N = 27)	4.58 (0.39)	5.44 (0.39)	5.26 (0.39)	5.30 (0.39)	5.22 (0.39)	.3151
	Marlboro Snus (N = 23)	3.78 (0.43)	4.83 (0.42)	5.09 (0.42)	5.74 (0.42)	5.39 (0.42)	.6883
	Stonewall (N = 24)	3.77 (0.42)	4.63 (0.41)	4.46 (0.41)	5.67 (0.41)	5.54 (0.41)	.0043
	Ariva (N = 24)	3.37 (0.42)	4.50 (0.41)	4.11 (0.42)	5.88 (0.42)	5.58 (0.41)	.0029
Concerned about dependence	Camel Snus (N = 27)	1.60 (0.26)	2.33 (0.26)	2.07 (0.26)	1.39 (0.26)	1.78 (0.26)	.0032
	Marlboro Snus (N = 23)	1.43 (0.28)	1.96 (0.28)	1.74 (0.28)	1.36 (0.28)	1.43 (0.28)	.3935
	Stonewall (N = 24)	1.33 (0.28)	1.63 (0.27)	1.67 (0.27)	2.17 (0.27)	1.71 (0.27)	.0089
	Ariva (N = 24)	1.67 (0.27)	2.13 (0.27)	1.96 (0.27)	2.16 (0.28)	2.21 (0.27)	.3002

Note. ^aHypothesis: the product chosen at the end of sampling period for subsequent intervention scored better than the average of the scores of the other four products on Product Evaluation Scale when those products were sampled during sampling period.

Relationship Between Subjective Responses and Extent of Product Use During the Treatment Period

During the 14-day smoking abstinence and product use (treatment) period, PES data were collected on Day 2, 7, and 14 visits, whereas amount of product use was recorded daily from Days 1 to 14. Visit-specific averaged amount of product use was first calculated and used as a repeatedly measured outcome. The association between PES and amount of product use was examined as an indication of validity using a general linear mixed model. The initial full model contained the four PES subscales and three individual items, and site, visit, product group, and the two interaction terms (product group × site and product group × visit) as the fixed effect, and random intercepts and random slopes for PES subscales and items as the random effects. Random slopes were included because the PES subscales and items collected repeatedly during the treatment period were time-varying covariates. Later, it was found that those random slopes carried very little variability; therefore, we considered random intercepts as the only random effect in the full model.

The results from the full model showed that satisfaction subscale (p = .021) was the only significant subjective response for product intake: one unit increase in satisfactory score was associated with 0.61 unit increase in the amount of product use, given the values of other covariates were fixed.

DISCUSSION

Factor analysis revealed that the subscales on the PES were similar to the mCEQ, with the following exceptions: (a) “did you enjoy sensations in the mouth” was added to the satisfaction subscale, (b) “was it too much nicotine” and “were there bothersome side effects” were added to the aversion subscale, (c) a common factor associated with craving and withdrawal relief was found (as a result of adding additional items to the mCEQ). Items for psychological award were identical to the mCEQ. The concordance of these factors with those observed for the mCEQ lends some validity to this scale.

Using these factors and the individual items, the results are similar to the ones described in a previous article in which subjective assessments, using a different scale, were made after sampling all the tobacco products (Hatsukami et al., 2011). That is, in both study analyses, the least positive subjective responses were associated with General Snus, which led to no subjects choosing this product for extended use. Most likely, General Snus was considered unpalatable because it is a Swedish manufactured product and not suited for the U.S. smoker’s palate. Only one other significant difference was observed across tobacco products. Dissolvable products were considered more comfortable for use than the snus products. These findings suggest that either this scale is not sensitive enough to capture differences across most oral tobacco products or for smokers, oral tobacco products generally do not substantially differ in their characteristics (e.g., they are all poorly rated).

Validity of the PES is highly supported by the relationships between subjective responses and product choice. In general, products that were chosen had higher mean ratings on satisfaction, withdrawal and craving relief and lower ratings on aversion. For those who chose higher level nicotine content products, ratings on psychological reward and “concerned about dependence” (indicating the addictiveness of the product) were significantly associated with product choice. For those who chose dissolvables, ease and comfort of use were the associated variables. These findings point to the notion that product appeal is related to certain common dimensions, but beyond that, individual preferences for such factors as reward from the product or ease of use may come into play.

Another measure of validity was how subjective responses are related to the amount of product use. The only item that demonstrated a significant relationship was satisfaction. This finding is similar to one that has been observed in our prior studies conducted with oral tobacco products and reduced nicotine content cigarettes (Schiller et al., 2012). These findings indicate either that the scale is limited in its ability to determine how much a product will be used or that a subject’s product satisfaction is the primary factor associated with amount of use. Clearly, more research needs to be conducted to understand the concept of satisfaction from a product.

Although other studies have been conducted using different scales to assess oral tobacco products (Blank & Eissenberg, 2010; Cobb et al., 2010; Gray et al., 2008; Kotlyar et al., 2007; Mendoza-Baumgart et al., 2007; O’Connor et al., 2011), none of these studies involved a systematic assessment of the validity of their scales. Some studies have shown that newer oral tobacco products compared with own brand tobacco products result in lesser levels of (a) craving and/or withdrawal relief, (b) direct effects of nicotine (i.e., feel more awake/alert, calm you down/relax, or help concentration) or of the tobacco product (i.e., taste good, satisfying, pleasant, product strength), (c) liking, or (d) desirability of the product (Blank & Eissenberg, 2010; Cobb et al., 2010; Gray et al., 2008; Kotlyar et al., 2007). Other studies have shown dose-related effects on such items as nausea (Blank, Sams, Weaver, & Eissenberg, 2008) and on measures of craving (Blank et al., 2008; Cobb et al., 2010; Kotlyar et al., 2007). Studies have also shown a relationship between subjective response to a product and choice. In the O’Connor (2011) study, the product that resulted in the highest frequency of individuals endorsing “liked most” (e.g., nicotine lozenge) after a product sampling phase was the one that was most frequently selected for use during single product choice phase. Similar observations were made for a study conducted by Mendoza-Baumgart et al. (2007), in which the product that was rated more desirable, likeable, or with the least bad effects during a sampling phase was the one that was selected during the product choice phase.

On the topic of experimental design, it is notable that we saw a carry-over effect for aversion. That is, if Ariva or Camel Snus were sampled first, then the next product sampled would be rated higher on aversion than if General Snus (a product rated most high on aversion) was sampled first. This finding indicates that the product sampling may need to be separated by more than 1 or 2 days.

In summary, specific items on the PES for oral tobacco products are associated with product choice and amount of product use. This scale will require more testing to determine if it is a valid tool to determine the abuse liability of an oral product and its utility for other noncombustible products.

FUNDING

This work was supported by the National Cancer Institute at the National Institutes of Health (R01 CA135884 and U19 CA157345).

DECLARATION OF INTERESTS

Dorothy Hatsukami was funded by Nabi Biopharmaceuticals and National Institute on Drug Abuse (NIDA) to be a site for a nicotine immunotherapy trial. There are no other declarations.