Comparison of Direct and Indirect Measures of E-cigarette Risk Perceptions

Abstract

Objectives

Risk perception measures of tobacco products relative to cigarettes are commonly used and important to tobacco research given that they may be associated with and predict tobacco use. However, results may differ based on measures used. This study compares direct and indirect approaches to measuring e-cigarette/cigarette risk perceptions.

Methods

We compared the responses of 519 current smokers on a nationally representative 2014 survey that gauged perceptions of e-cigarettes’ harm relative to cigarettes in two ways: 1) a single-item direct measure of comparative harm and a two-item indirect measure (which measured perceived levels of harm from e-cigarettes and cigarettes independently in two parallel questions).

Results

We found that 60% of smokers rated e-cigarettes “less harmful” than cigarettes when using a direct comparative risk measure versus 73% when using an indirect measure. Agreement between measure types was fair (Cohen’s Kappa=0.45) and was lower for males, Blacks, older and less educated smokers.

Conclusions

E-cigarettes were more likely to be rated by smokers as less harmful than cigarettes when using indirect versus direct measures. Additional methodology research into this area is warranted given the importance of risk perceptions to tobacco control interventions, communications, policy-making and regulation.

INTRODUCTION

Study of tobacco product risk perceptions is important for understanding tobacco use patterns and informing tobacco regulatory policy given that they may be associated with and predict tobacco use. For example, in facing high cigarette taxes, smoking bans, and health concerns, smokers may look for and choose alternatives they believe pose a comparable if not lower risk. Indeed, several studies have pointed to beliefs that e-cigarettes are safer than cigarettes as a major reason given by smokers for trying or switching to e-cigarettes.^1–5 In addition, former or never smokers may initiate use of a new product they believe is enjoyable but safer than smoking. A longitudinal study with young adults found that those at baseline who believed e-cigarettes were less harmful than tobacco cigarettes, including former smokers and non-smokers, were more likely to have used e-cigarettes at follow up.⁶ On the other hand, it has also been argued that having inaccurate perceptions about the risks of different products relative to cigarettes might do harm by preventing smokers from switching to a less harmful product.^7–9 For these various reasons, tobacco surveillance surveys sometimes include some measurement of the perceived comparative risks of different tobacco products.¹⁰

Surveys typically measure these risk perceptions with a single direct measure of comparative harm (eg, “Compared to regular cigarettes, is ___ tobacco product less harmful, as harmful or more harmful?”).^11–16 However, recent research has suggested that different results may be obtained when using a two-item indirect measure instead (ie, measuring and comparing perceived levels of harm for cigarettes and another tobacco product in two parallel questions). Popova and Ling (2013) included both direct and indirect measures for assessing perceived harm from snus (a low nitrosamine form of smokeless tobacco [SLT]) versus cigarettes in a single survey.¹⁷ Only 22.1% of respondents rated snus as being less harmful than cigarettes when measured directly compared to 51.6% who thought so with the indirect measures. This suggested that comparative risk measures may underestimate individuals’ perceptions that SLT is less harmful relative to cigarettes, and raises questions as to whether indirect risk measures may better assess individuals’ true SLT risk perceptions.¹⁷

We explored this measurement issue with respect to another new “cigarette alternative”, e-cigarettes, which have rapidly grown in popularity and public awareness since their 2006–2007 introduction, and to date are largely perceived as being less harmful than cigarettes using direct measures.^{6,11,14,16,19}

METHODS

We conducted an online survey in April, 2014 with 519 current smokers recruited from GFK’s Knowledge Networks research panel. GFK panel members are recruited through probability-based sampling of addresses from the US Postal Service’s Delivery Sequence File and samples for individual surveys are drawn from the panel using a probability proportional to size weighted sampling approach to result in nationally representative samples. Details about GFK’s recruitment and sample methodology are available online.²⁰ For the current study, GFK sampled 1,042 participants from its panel, of which 609 (58.4%) completed the study screener (ie, inclusion criteria defined as having ever smoked 100 cigarettes and now smoking “some days” or “everyday”) and 519 subsequently qualified for and completed the survey. The weighted demographics of our sample (sex, race, age, education, census region) were comparable to current smokers in the 2013 National Health Interview Survey. The survey asked respondents questions about their e-cigarette awareness, information sources, use (including intentions and reasons), risk perceptions and policy attitudes. Additional details and other results have been reported elsewhere.^21,22

Measures used for assessing comparative e-cigarette/cigarette risks directly and indirectly were adapted from Popova & Ling’s study (2013).¹⁷ All participants answered an indirect question about the perceived harm of cigarettes (“In your opinion, how harmful is smoking cigarettes to one's health?”) early in the survey in a general section about respondents’ smoking use and history. Respondents provided answers on a Likert scale of 1 (“not at all harmful”) to 7 (“extremely harmful”).

Later in the survey (after e-cigarette awareness and ever-use questions), participants were asked indirect and direct e-cigarette harm perceptions questions. The order (ie, indirect first or last) was randomized and the 2 questions were separated from each other by 5 others. The direct harm measure asked, “In your opinion, compared to smoking regular cigarettes, how harmful is using electronic cigarettes?”, using a 7 point Likert scale of −3 (“a lot less harmful”) to 0 (“equally harmful”), to +3 (“a lot more harmful”). The indirect e-cigarette harm question mirrored the cigarette question (“...how harmful is using e-cigarettes to one's health?”). During analysis, we compared responses to the two parallel indirect measures, coding the same rating provided for both as an “as harmful” perception, a lower rating for e-cigarettes as “less harmful”, and a higher rate for e-cigarette than cigarettes as “more harmful.”

Weighted prevalence estimates and 95% confidence intervals are presented and a kappa statistic was calculated to assess agreement between the direct and indirect measures. Following the guidelines proposed by Fleiss (1981),²³ poor, fair to good, and excellent agreement ratings were indicated by kappa statistics of less than 0.4, 0.4 to 0.75, and greater than 0.75, respectively. All analyses were conducted using SAS-Callable SUDAAN (Release 11.0.1), which accounts for complex sample designs.

RESULTS

The study sample (N = 519) was approximately equally distributed by sex (51% male, 49% female), 31.5% were young adults (ages 18–34), 66.1% were white (14.7% black, 12.7% Hispanic, 6.4% other), 40.1% had at least some college education, and 54% were currently employed. While all participants were current cigarette smokers, 80.3% reported daily smoking and 18.9% reported using e-cigarettes in the last 30 days.

The mean perceived harm was 5.9 (SD=0.07) for cigarettes and 3.9 (SD=0.09) for e-cigarettes, with a significant mean difference of 2.0 (SD=0.10, p < .0001). Table 1 highlights risk perceptions by measure type − 60% of smokers rated e-cigarettes as less harmful than cigarettes with the direct measure versus 73% with the indirect measure. Agreement between measure types was fair (Cohen’s Kappa= 0.45; 0.36–0.54, p < .0001) – 93.9% of smokers who directly ranked e-cigarettes as less harmful also indirectly rated e-cigarettes as less harmful than regular cigarettes, but approximately 36% who directly ranked e-cigarettes as more harmful than regular cigarettes indirectly rated them as less harmful. Agreement was lower for those aged 60+, males, Blacks, and those less educated (see Table 2).

Table 1.

Percentage of Smokers Rating E-cigarettes as being Less Harmful, As Harmful or More Harmful than Cigarettes using Indirect versus Direct Measures

	Using Indirect Measures
	E-cigs less harmful		Same harm		E-cigs more harmful		Total
	%	(95% CI)	%	(95% CI)	%	(95% CI)	%	(95% CI)
Using Direct Measures
E-cigs less harmful	93.9	(90.1 –96.3)	5.6	(3.2 –9.4)	0.5	(0.1 –2.1)	60.0	(54.6 –65.1)
Same harm	48.8	(39.2 –58.4)	47.8	(38.3 –57.5)	3.4	(1.4 –8.4)	28.8	(24.2 –33.8)
E-cigs more harmful	36.1	(22.7 –52.2)	36.2	(22.5 –52.6)	27.7	(14.8 –45.8)	11.3	(8.3 –15.2)
Total	73.0	(67.9 –77.5)	22.7	(18.5 –27.5)	4.4	(2.6 –7.3)	100	……

Abbreviations: E-cig, Electronic cigarette; CI, Confidence Interval

Table 2.

Comparison of Direct and Indirect Measures of E-Cigarette Harmfulness Relative to Traditional Cigarettes: Kappa Agreement Statistics by Demographic, Smoking, and E-Cigarette Characteristics

	Kappa	(95% CI)
Overall	0.45	(0.36 –0.54)
Age
18–29	0.45	(0.19 –0.71)
30–44	0.48	(0.33 –0.64)
45–59	0.47	(0.34 –0.61)
60+	0.34	(0.16 –0.51)
Sex
Male	0.39	(0.27 –0.50)
Female	0.52	(0.40 –0.64)
Race/Ethnicity
Non-Hispanic white	0.48	(0.38 –0.58)
Non-Hispanic black	0.27	(0.04 –0.50)
Hispanic	0.47	(0.19 –0.75)
Non-Hispanic other/multiple races	0.51	(0.27 –0.75)
Education
High school or less	0.38	(0.27 –0.50)
Some college or more	0.56	(0.44 –0.68)
Smoking Frequency
Daily	0.45	(0.36 –0.54)
Some days	0.45	(0.24 –0.67)
Cigarettes per day
10 or less	0.42	(0.30 –0.54)
11 or more	0.48	(0.36 –0.60)
Plan to quit within 6 months
Yes	0.41	(0.28 –0.55)
No	0.58	(0.37 –0.59)
E-Cigarette Use
Current user1	0.38	(0.14 –0.62)
Former user2	0.52	(0.38 –0.66)
Never user	0.38	(0.25 –0.51)

Abbreviations: CI, Confidence Interval

¹Defined as e-cigarette use in the past 30 days

²Defined as ever using e-cigarettes but not in past 30 days

DISCUSSION

This is the first study to compare direct and indirect approaches to measuring e-cigarette/cigarette risk perceptions. Consistent with Popova and Ling’s (2013) snus study,¹⁷ we found that e-cigarettes were more likely to be rated as less harmful than cigarettes when using indirect versus direct measures. Popova and Ling suggested that direct relative risk measures may underestimate individuals’ perceptions that smokeless tobacco products are less harmful than cigarettes because respondents may be more likely to provide what they think is the socially correct or desired answer (ie, that SLT and cigarettes are “equally” harmful) when asked about the risks of SLT versus cigarettes in the same question. However, our observed differences (approximately 13 percentage points) were not as great as Popova & Ling’s (approximately 30 percentage points). This difference might be attributed to a greater social desirability to provide perceived “correct” answers (ie, that products are equally harmful) for smokeless tobacco versus e-cigarettes, given that SLT products have known health risks and have traditionally been called a “not safe smoking alternative” in warning labels and other educational messages. In contrast, the health risks of e-cigarettes have not yet been clearly established and e-cigarettes are not yet required to carry any warning messages. Furthermore, e-cigarettes have both actively and implicitly been endorsed as being less harmful alternatives in advertising.^24,25

However, reasons other than social desirability bias may account for differences in responses between direct and indirect measures. For example, consumer research data suggest that some items (or their attributes) may be difficult to evaluate without some context for comparison and that this issue can account for differences in ratings of items compared directly or evaluated separately.²⁶ It may be that it is more difficult for smokers to rate their risk perceptions of some products they are less experienced or familiar with (like smokeless tobacco products) without having some reference product in the question to directly compare to.

Additional methodology research into this area is warranted given the importance of risk perceptions to tobacco control interventions, communications, policy-making and regulation. In addition, this study sample only included adult current smokers, and thus does not reflect agreement between risk perception types between other potential users of e-cigarettes, including non-smokers, former smokers and youth, who may or may not have similar patterns of responses. Future research should also explore this issue with these audiences.

It should also be noted that both the indirect and direct measures described in this paper are limited in being simplistic measures of tobacco product risk perceptions. Risk perception literature suggests that people judge risks on a variety of dimensions such as dread, familiarity, and number of people exposed,²⁷ that they may moderate their risk perceptions based on beliefs about those products’ benefits and voluntary use,²⁷ and that risk measures should assess various specific harms to health rather than just harm in general.^28,29 As such, tobacco risk perceptions should ideally be measured with more than one item.²⁹ One recent study that did this found that while smokers consistently believed e-cigarettes to be less likely to cause lung cancer, heart disease and oral cancer than tobacco cigarettes, they only perceived smokeless tobacco as being less harmful than cigarettes with respect to causing lung cancer.⁹ In practice however, large national and state tobacco surveillance surveys may not be able to include multiple risk perception measures for various tobacco products given time and budget considerations. One option could be to have respondents complete a more comprehensive set of risk perception measures for cigarettes and only one other non-cigarette product (like e-cigarettes or snus) that they are randomly assigned to.⁹ However, this approach runs the risk of reducing power to find significant differences between product types because of reduced sample sizes. Another option might be to pursue psychometric research aimed at determining whether one or two particular risk measures (out of a full battery of possible risk measures) can account for the majority of variance in risk perception scores, and/or are the best predictors of future tobacco use intentions or behaviors (eg, initiation, cessation, product switching).

IMPLICATIONS FOR TOBACCO REGULATION

Tobacco risk perception research is important and highly relevant to the Food and Drug Administration’s (FDA) tobacco regulatory authority, including its expected regulation over e-cigarettes.³⁰ Indeed, the Tobacco Control Act gave the FDA authority to regulate how tobacco products covered under the law may be labeled and communicated. Since these policies may impact tobacco risk perception beliefs and subsequent product use, relevant science is needed to inform them. The FDA also recognizes that different tobacco products may fall onto a continuum of differing levels of risk^31,32 and is accepting applications from tobacco companies to market their products as being “modified” or “lower risk”.³¹ Already one company has applied for modified risk designation of its snus brand (though not approved to date)³³ and future applications to market e-cigarettes as reduced risk seem likely once e-cigarettes are under FDA’s authority. If approved, post-market surveillance of these products and the impact of their labeling on risk perceptions and product use is required, and may result in label revocation or modification if it is found to produce net population harm rather than benefit (eg, by attracting more new product users versus people switching from cigarettes to the alternative product).³¹ As such, having appropriate tobacco risk perception measures, including for e-cigarettes, is and will be highly relevant for providing context to tobacco use patterns and for informing these regulations.