Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Apr 1.
Published in final edited form as: Addict Behav. 2017 Jan 4;67:100–105. doi: 10.1016/j.addbeh.2017.01.001

Criterion Validity of Measures of Perceived Relative Harm of E-Cigarettes and Smokeless Tobacco Compared to Cigarettes

Alexander Persoskie 1, Anh B Nguyen 1, Annette R Kaufman 2, Cindy Tworek 1
PMCID: PMC5257335  NIHMSID: NIHMS841910  PMID: 28073035

Abstract

Beliefs about the relative harmfulness of one product compared to another (perceived relative harm) are central to research and regulation concerning tobacco and nicotine-containing products, but techniques for measuring such beliefs vary widely. We compared the validity of direct and indirect measures of perceived harm of e-cigarettes and smokeless tobacco (SLT) compared to cigarettes. On direct measures, participants explicitly compare the harmfulness of each product. On indirect measures, participants rate the harmfulness of each product separately, and ratings are compared. The U.S. Health Information National Trends Survey (HINTS-FDA-2015; N=3738) included direct measures of perceived harm of e-cigarettes and SLT compared to cigarettes. Indirect measures were created by comparing ratings of harm from e-cigarettes, SLT, and cigarettes on 3-point scales. Logistic regressions tested validity by assessing whether direct and indirect measures were associated with criterion variables including: ever-trying e-cigarettes, ever-trying snus, and SLT use status. Compared to the indirect measures, the direct measures of harm were more consistently associated with criterion variables. On direct measures, 26% of adults rated e-cigarettes as less harmful than cigarettes, and 11% rated SLT as less harmful than cigarettes. Direct measures appear to provide valid information about individuals’ harm beliefs, which may be used to inform research and tobacco control policy. Further validation research is encouraged.

Keywords: perceived harm, measures, validity, tobacco, electronic cigarettes, smokeless tobacco

1. Introduction

Beliefs about harm are important determinants of health behavior (1-4) such as tobacco and nicotine product use. For example, a systematic review of research on electronic cigarette (e-cigarette) use identified the belief that e-cigarettes are less harmful than cigarettes as a common reason for using e-cigarettes (5). Some studies suggest that U.S. adults tend to overestimate the harms of e-cigarettes and smokeless tobacco (SLT) compared to cigarettes, which may discourage smokers from switching to less harmful alternatives (6-8). Low perceptions of harm may also encourage uptake among non-users or prevent cessation among current users of products (9-11). Due to practical and theoretical importance, harm beliefs are often the focus of public health research and educational campaigns, and are used to assess the impacts of marketing, advertising, and tobacco control policies (6-8, 12-19).

The choice of measures can strongly influence research results on perceived harm (20-22), but, despite the importance of measuring harm beliefs, public health researchers lack standard, well-validated measures. Two types of measures include direct measures (e.g., where people rate the harm of using e-cigarettes as lower or higher than the harm of using cigarettes in a single question) and indirect measures (e.g., where people rate the harms of using e-cigarettes and cigarettes on two separate questions, and then ratings are compared to determine whether the harm for one product was rated lower than the other). Studies indicate that cigarette smokers are more likely to rate e-cigarettes and SLT as less harmful than cigarettes on indirect rather than direct measures (20, 21). In one study, nearly 52% of smokers rated snus – a type of SLT – as less harmful than cigarettes on an indirect measure compared to only 22% on a direct measure (20). Moreover, direct and indirect measures of perceived harm may correlate only moderately (e.g., r=0.3) (20).

Researchers have hypothesized that direct measures may bias people toward giving a “socially appropriate answer” (i.e., that non-combustible products are just as harmful as cigarettes) (20, 21), thus underestimating the extent to which people believe non-combustible products are less harmful than cigarettes. However, no studies have tested this hypothesis to examine whether one type of measure more accurately reflects public perceptions than the other, or whether each type may reflect a unique aspect of harm perception.

This study compared the construct validity of direct and indirect measures of perceived harm of e-cigarettes and SLT compared to cigarettes. Construct validity–described informally as the accuracy of a measure–refers to the extent to which a measure assesses the construct it is intended to assess (23). We focused on an aspect of construct validity called criterion validity, or “the extent to which a measure is empirically associated with relevant criterion variables” (24). In studies evaluating criterion validity, the criterion variables are selected based on theoretical predictions about how a measure should be associated with them (e.g., researchers studying nicotine dependence would want to show that the measure correlates negatively with smokers’ likelihood of successfully quitting smoking) (24, 25).

We expected perceived relative harm of products to be associated with product use behavior. This expectation was based on the known effects of perceived harm on behavior (e.g., people who perceive little harm in using a product tend to be more likely to try it) and the known effects of behavior on perceived harm (e.g., people who use a product become more likely to perceive little harm in using it) (9, 18, 26-30). Indeed, public health interest in harm beliefs is predicated on the notion that harm beliefs have implications for product trial and use (8, 12, 13, 22, 29-31). Therefore, people who have tried a particular non-cigarette product should, in theory, be more likely than others to believe that the product is less harmful than cigarettes (1, 9, 26-30). Thus, we selected as criterion variables measures of product use, including ever-trying e-cigarettes, ever-trying snus, and current and former use of SLT, and we examined whether direct and indirect measures of relative harm were associated with these variables. When the expected associations emerged between a measure of relative harm and the criterion variables, we took this as evidence in favor of the validity of the measure (24, 25).

2. Methods

2.1. Data Source

The National Cancer Institute's Health Information National Trends Survey (HINTS-FDA 2015) is a cross-sectional mail survey assessing health-related beliefs and behaviors. HINTS-FDA 2015 is nationally representative of the non-institutionalized U.S adult population. Data were collected between May and September 2015. Households were selected using a random sample of U.S. addresses; within households, one adult was selected based on proximity of birthdate to survey date. A complex sampling design was employed, with an effort to oversample residential strata with high proportions of current and former cigarette smokers. The overall weighted response rate for HINTS-FDA 2015 was 33%. In total, 3738 individuals returned eligible surveys. Additional details can be found elsewhere (32, 33).

2.2. Measures

2.2.1. Direct and Indirect Measures of Perceived Relative Harm

The direct measure of e-cigarette relative harm stated, “Compared to smoking cigarettes, would you say that electronic cigarettes are...” Options included “Much less harmful,” “Less harmful,” “Just as harmful,” “More harmful,” “Much more harmful,” “I've never heard of electronic cigarettes,” and “I don't know enough about these products.” Participants selecting “Much less harmful” or “Less harmful” were coded as rating e-cigarettes lower than cigarettes. Those selecting any other option were coded as not rating e-cigarettes as lower than cigarettes (27).

The direct measure of SLT relative harm stated, “In your opinion, do you think that some smokeless tobacco products, such as chewing tobacco, snus and snuff, are less harmful to a person's health than cigarettes?” Options were “Yes,” “No,” and “Don't know.” Participants selecting “Yes” were coded as rating SLT as less harmful than cigarettes. Those selecting any other option were coded as not rating SLT as less harmful than cigarettes.

Indirect measures of e-cigarette and SLT relative harm were created based on the question: “How harmful do you think each of the following is to a person's health?” Participants separately rated the harms of “Cigarette smoking,” “Electronic cigarette use,” and “Smokeless tobacco use,” each on a single item with options: “Not at all harmful,” “Moderately harmful,” and “Very harmful.” Participants were coded as rating e-cigarettes as less harmful than cigarettes if their rating for “Electronic cigarette use” was lower than their rating for “Cigarette Smoking,” and were coded as not rating e-cigarettes lower than cigarettes if their rating for “Electronic cigarette use” was equal to or higher than their rating for “Cigarette Smoking.”1 Similarly, participants were coded as rating SLT as less harmful than cigarettes if their rating for “Smokeless tobacco use” was lower than their rating for “Cigarette Smoking,” and as not rating SLT lower than cigarettes if their rating for “Smokeless tobacco use” was equal to or higher than their rating for “Cigarette Smoking.”2

Perceived harm measures were located together on the survey with indirect measures directly preceding the direct measures.

2.2.2. Criterion Variables

Criterion variables included all relevant product use variables in HINTS-FDA 2015: ever-trying e-cigarettes, ever-trying snus, and SLT use status.3

Ever-trying e-cigarettes and ever-trying snus were assessed with: “Which of the following tobacco products have you ever tried even once?” Participants were coded as having tried e-cigarettes if they selected “Electronic Cigarettes (such as blu, NJOY or Logic), also known as vape-pens, hookah pens, e-hookahs, or e-vaporizers.” Participants were coded as having tried snus if they selected “Snus (such as Camel snus, General snus, Marlboro snus, and Nordic Ice).”

SLT use status (current, former, never) was assessed with two questions: “Have you used chewing tobacco, snus, snuff, or dip, at least 20 times in your entire life? Some popular brands include Redman, Levi Garrett, Beechnut, Skoal, or Copenhagen” (Yes/No), and “Do you now use chewing tobacco, snus, snuff, or dip every day, some days, or not at all?” (Every day/Some days/Not at all) Participants were coded as current SLT users if they reported using SLT at least 20 times in their lives and currently using “Every day” or “Some days,” as former SLT users if they reported using SLT at least 20 times but currently using “Not at all,” and as never-users if they reported using SLT less than 20 times (34).

2.3. Statistical Analyses

Criterion validity was assessed by testing whether direct and indirect measures of perceived relative harm were associated with the criterion variables, where relevant4: ‘ever-tried e-cigarettes,’ ‘ever-tried snus,’ and SLT use status. This was accomplished through a series of weighted binary and multinomial logistic regressions, as appropriate to each criterion variable. Each analysis had two steps. First, the criterion variable was regressed separately on the direct and indirect measures of perceived relative harm (e.g., ‘ever-tried e-cigarettes’ was regressed on the direct measure of e-cigarette perceived harm and then on the indirect measure). This analysis revealed whether each perceived harm measure was associated with the criterion variable without controlling for the other perceived harm measure. Next, each criterion variable was regressed on the direct and indirect measures simultaneously to determine whether the direct and indirect measures explained unique variance in the criterion variable (e.g., ‘ever-tried e-cigarettes’ was regressed simultaneously on both the direct and indirect measures of e-cigarette perceived harm).

Analyses were weighted to account for the study's complex sampling design (32). Following standard practice in criterion validation, we did not adjust for sociodemographic characteristics, as the aim was not to test whether perceived harm was associated with behavior but to compare criterion validity of perceived harm measures (see, e.g., 25). Statistical significance was set at p<.05. No imputation strategy was used to fill missing data; respondents were excluded from analyses in which they had missing data on one or more variable. Analyses were conducted in SAS 9.4 and SAS-callable SUDAAN Version 11.0 (RTI International, Research Triangle Park, North Carolina).

3. Results

3.1. Sample Characteristics

The sample was about half female (50.9%; n=2018) with an average age of 47.2 (SE=0.13). Educational attainment included: high school diploma or less (31.9%; n=964), some college or vocational training (32.8%; n=1132), college degree (20.3%; n=906), and postgraduate training (15.0%; n=672).

3.2. Perceived E-Cigarette Relative Harm

People were less likely to rate e-cigarettes as less harmful than cigarettes on the direct measure (26.3%; 95% CI=24.0-28.7) than on the indirect measure (44.8%; 95% CI=41.6-47.9).5 Direct and indirect measures of perceived relative e-cigarette harm were only moderately associated (r [phi coefficient]=0.36; 95% CI=0.29-0.42). People rating e-cigarettes as less harmful than cigarettes on the direct measure were more likely to also do so on the indirect measure (74.1% vs. 34.0%), OR=5.55, 95% CI=3.85-8.00, p<.0001, Cox & Snell R2=0.12. Nearly half (44.7%) of people who rated e-cigarettes as less harmful than cigarettes on the indirect measure did not do so on the direct measure.

When ‘ever-trying e-cigarettes’ was regressed separately on the direct and indirect measures of e-cigarette relative harm, significant associations emerged for both measures (Table 1, Models 1 and 2). However, the direct measure was more strongly associated with ever-trying e-cigarettes than the indirect measure: People who rated e-cigarettes as less harmful than cigarettes on the direct measure were 2.7 times more likely to have tried an e-cigarette (38.6% vs. 14.4%), whereas people who rated e-cigarettes as less harmful on the indirect measure were 1.7 times more likely to have tried an e-cigarette (27.7% vs. 16.6%). When both measures were entered simultaneously in the same model (Table 1, Model 3), the direct measure was significantly associated with ever-trying e-cigarettes while the indirect measure was not.

Table 1.

Association between e-cigarette trial and ratings of e-cigarettes as less harmful than cigarettes on direct and indirect measures.

Tried an E-Cigarette n (weighted %)
No Yes OR (95% CI)* p Cox & Snell R2
Model 1 .064
    Direct Measure: E-Cigarettes Rated Less Harmful
        No 2102 (85.6) 232 (14.4) 1.00 (Ref)
        Yes 569 (61.4) 249 (38.6) 3.74 (2.67,5.24) <.0001
Model 2 .018
    Indirect Measure: E-Cigarettes Rated Less Harmful
        No 1437 (83.4) 176 (16.6) 1.00 (Ref)
        Yes 1150 (72.3) 305 (27.7) 1.92 (1.33,2.77) .0008
Model 3 .066
    Direct Measure: E-Cigarettes Rated Less Harmful
        No ... ... 1.00 (Ref)
        Yes ... ... 3.44 (2.38,4.97) <.0001
    Indirect Measure: E-Cigarettes Rated Less Harmful
        No ... ... 1.00 (Ref)
        Yes ... ... 1.24 (0.83,1.84) .290
Open in a new tab
*

OR = Odds ratio from binary logistic regression of e-cigarette trial on the direct measure (Model 1), indirect measure (Model 2), and the direct and indirect measures entered simultaneously (Model 3). As is standard practice in criterion validation studies, models did not adjust for other variables.

3.3. Perceived SLT Relative Harm

People were less likely to rate SLT as less harmful than cigarettes on the direct measure (11.0%; 95% CI=9.2-12.7) than on the indirect measure (25.5%; 95% CI=22.7-28.4).6 These two measures were weakly associated (r [phi coefficient]=0.18; 95% CI=0.10-0.26). People rating SLT as less harmful than cigarettes on the direct measure were more likely to also do so on the indirect measure (47.9% vs. 22.8%), OR=3.11, 95% CI=1.98-4.91, p<.0001, Cox & Snell R2=0.03. Nearly four-fifths of people (79.4%) who rated SLT as less harmful than cigarettes on the indirect measure did not do so on the direct measure.

When ‘ever-tried snus’ was regressed separately on direct and indirect measures of SLT relative harm (Table 2, Models 1 and 2), a significant association emerged for the direct measure and a marginal association emerged for the indirect measure (p=.050). The association was stronger for the direct measure than the indirect measure: People who rated SLT as less harmful than cigarettes on the direct measure were 1.99 times more likely to have tried snus (17.9% vs. 9.0%), and those who rated SLT as less harmful on the indirect measure were 1.60 times more likely to have tried snus (13.6% vs. 8.5%). When ‘ever-tried snus’ was regressed simultaneously on both measures, neither measure was significantly associated with snus trial (Table 2, Model 3).

Table 2.

Association between snus trial and ratings of SLT as less harmful than cigarettes on direct and indirect measures.

Tried Snus n (weighted %)
No Yes OR (95% CI)* p Cox & Snell R2
Model 1 .007
    Direct Measure: SLT Rated Less Harmful
        No 2669 (91.0) 147 (9.0) 1.00 (Ref)
        Yes 353 (82.1) 43 (17.9) 2.21 (1.11,4.39) .025
Model 2 .005
    Indirect Measure: SLT Rated Less Harmful
        No 2155 (91.5) 124 (8.5) 1.00 (Ref)
        Yes 763 (86.4) 61 (13.6) 1.69 (1.00,2.87) .050
Model 3 .011
    Direct Measure: SLT Rated Less Harmful
        No ... ... 1.00 (Ref)
        Yes ... ... 1.99 (0.91,4.33) .082
    Indirect Measure: SLT Rated Less Harmful
        No ... ... 1.00 (Ref)
        Yes ... ... 1.50 (0.84,2.67) .165
Open in a new tab
*

OR = Odds ratio from binary logistic regression of snus trial on the direct measure (Model 1), indirect measure (Model 2), and the direct and indirect measures entered simultaneously (Model 3). As is standard practice in criterion validation studies, models did not adjust for other variables.

When SLT use status was regressed separately on direct and indirect measures of SLT relative harm (Table 3, Models 1 and 2), significant associations emerged for the direct measure but not the indirect measure. People who rated SLT as less harmful than cigarettes on the direct measure were more likely to have used SLT: They were 3.2 times more likely to be current SLT users (6.8% vs. 2.1%) and 2.2 times more likely to be former SLT users (13.6% vs. 6.2%). When SLT use status was regressed simultaneously on direct and indirect measures (Table 3, Model 3), only the direct measure was associated with current and former SLT use.

Table 3.

Association between SLT use status and ratings of SLT as less harmful than cigarettes on direct and indirect measures.

SLT Use Status n (weighted %)
Never Former Current
Model 1
    Direct Measure: SLT Rated Less Harmful
        No 2947 (91.7) 188 (6.2) 53 (2.1)
        Yes 359 (79.5) 53 (13.6) 30 (6.8)
            OR [95% CI] (ref) 2.54 [1.12,5.80] 3.68 [1.82,7.46]
Model 2
    Indirect Measure: SLT Rated Less Harmful
        No 2344 (90.9) 177 (6.5) 45 (2.6)
        Yes 818 (87.8) 60 (8.9) 36 (3.3)
            OR [95% CI] (ref) 1.42 [0.79,2.55] 1.33 [0.73,2.45]
Model 3
    Direct Measure: SLT Rated Less Harmful
        No ... ... ...
        Yes ... ... ...
            OR [95% CI] (ref) 2.46 [1.13,5.34] 3.68 [1.76,7.70]
    Indirect Measure: SLT Rated Less Harmful
        No ... ... ...
        Yes ... ... ...
            OR [95% CI] (ref) 1.23 [0.73,2.08] 1.05 [0.54,2.06]
Open in a new tab

*OR = Odds ratio from multinomial logistic regression of SLT use status on the direct measure (Model 1), indirect measure (Model 2), and the direct and indirect measures entered simultaneously (Model 3). ORs are significant at p<.05 if the 95% confidence interval does not include 1.00. As is standard practice in criterion validation studies, models did not adjust for other variables.

3.4. Sensitivity Analyses

The direct measures of e-cigarette and SLT relative harm each had “Don't know” (DK) response options, whereas the indirect measures did not. Thus, people who did not have well-defined beliefs about product harm did not have a chance to opt out of the indirect items by choosing a DK option. This difference may have affected the relative performance of the direct and indirect measures in terms of their associations with criterion variables.

To evaluate the potential confounding influence of the DK options, we replicated the above analyses after excluding DK responders on the direct measure for each product. In analyses of ever-trying e-cigarettes, we excluded from all analyses any participant responding either “I’ve never heard of electronic cigarettes” or “I don't know enough about these products” on the direct measure (n=1374; 35.6%). In analyses of ever-trying snus and SLT use status, we excluded any participant responding “Don't know” on the direct measure (n=796; 22.2%).

We found no evidence that doing so increased the criterion validity of the indirect measures compared to the direct measures. Results were consistent in direction and significance with those reported above, with one exception: For analyses of SLT use status, the association between the direct measure of SLT harm and former SLT use became non-significant when entered in models with and without the indirect SLT harm measure (OR=2.10, p=.078, and OR=2.02, p=.078, respectively). However, the association between the direct measure and current SLT use remained significant in both models, and all associations for the indirect measure remained non-significant.

4. Discussion

Because direct and indirect measures produce different findings, Popova and Ling (2013) suggested that researchers and regulators “should consider both direct and indirect measures when perceived risk data are presented as evidence for tobacco regulation” (20). Indeed, people are more likely to rate non-combustible products as less harmful than cigarettes on indirect rather than direct measures, with our findings revealing a 1.7-fold difference between indirect and direct measures for e-cigarettes (44.8% vs. 26.3%) and a 2.3-fold difference for SLT (25.5% vs. 11.0%). Researchers have hypothesized that such differences may arise because direct measures produce bias towards a socially desirable answer that e-cigarettes and SLT are just as harmful as cigarettes (20). If true, direct measures would have lower validity than indirect measures. In contrast with this hypothesis, tests of associations with criterion variables suggested that the direct measures had higher criterion validity compared to the indirect measures: In unadjusted models, direct measures were significantly associated with all criterion variables (ever-trying e-cigarettes and snus; SLT use status). In contrast, the indirect measure of SLT harm was not significantly associated with ever-trying snus or SLT use status, and the size of the association between perceived e-cigarette harm and ever-trying e-cigarettes was smaller for the indirect measure compared to the direct measure.

Entering the direct and indirect measures simultaneously in the same model assessed whether each measure of relative harm was uniquely associated with each criterion variable (and thus whether each measure may reflect a unique aspect of harm perception that should be assessed in public health research). If each measure explained unique variance in the criterion variable above and beyond the other, this would support the recommendation that studies should include both direct and indirect measures (20). For two of the three criterion variables (ever-trying e-cigarettes; SLT use status), models with both measures entered simultaneously revealed significant associations between the criterion variable and the direct measure but not the indirect measure.

These results suggest that the indirect measures had some degree of validity but were outperformed by–and did not provide information beyond–the direct measures. It is worth noting that the direct measures explicitly asked people to compare one product to another, whereas the indirect measures asked people to rate products separately and assumed they used the scale consistently across products, which may not be true. Also, for the direct measures, people may not have gone through the mental process of separately estimating the absolute harm of each product and then comparing them, as on the indirect measures. Rather, they may have given gut-level “gist” impressions of the type that tend to be influential in behavior (35) or they may have thought of reasons why one product is more or less harmful than the other. In direct comparisons, people may focus more on the object being evaluated than on the comparison object, which can also influence ratings (36). Analogous to the present findings, a prior study of comparative risk perceptions (i.e., perceptions of a risk to oneself compared to the risk to another person) found that direct comparisons tended to be more strongly associated with intentions to reduce one's risk than were indirect comparisons (37).

Further measurement validation work is encouraged given this study's limitations. The indirect items consisted of only three response options (“Not at all harmful”/“Moderately harmful”/“Very harmful”). Providing additional options may have allowed for finer-grained expressions of perceived harm, possibly translating into more valid indirect measures (38). Indeed, indirect measures used in prior studies have used more nuanced 7-point items (“not at all harmful” to “extremely harmful”). It is worth noting that the lower validity of the indirect measures observed here does not appear to have been caused by a ceiling effect, as people were more likely to rate e-cigarettes and SLT as less harmful than cigarettes on indirect measures.

The direct measures also had “Don't know” (DK) options whereas the indirect measures did not. In general survey design, evidence suggests that DK options do not tend to improve measurement validity (38). However, it is possible that DK responses may have special significance on risk perception items, as people may react strongly to hazards they perceive as unknown (39). Indeed, in one study, smokers responding DK on e-cigarette risk perception items were less likely to have tried e-cigarettes than were smokers who rated e-cigarettes as less risky than cigarettes (40). Although our sensitivity analyses found no evidence that the presence of DK options on the direct measures explained our results, we cannot rule out that adding a DK option may have improved the validity of the indirect measures.

This study used a two-level coding of perceived harm (“Less harmful” vs. any other option) (27, 41) rather than a 3-level coding (“Less harmful”/“Equally harmful”/“More harmful”) (16, 17, 20, 21). This prevented us from distinguishing participants who rated products as equally vs. more harmful than cigarettes. This was done to create direct and indirect measures that had the same levels, as not all measures had “More harmful” and “Don't know” options; also, there were small samples (n<40) of those rating e-cigarettes and SLT as more harmful than cigarettes on the indirect measures. Even in studies using alternate coding schemes, interpretation of findings has focused on individuals rating e-cigarettes and SLT as less harmful than cigarettes (8, 16, 17, 21, 30).

This study also included only a single measure of ever-trying e-cigarettes, lacking additional measures of use, and measures of product use were taken concurrently with measures of perceived harm. Further measurement validation work may use more detailed measures of product use and may examine whether current measures of perceived harm predict future product use. The analysis evaluated two types of direct measures, including a rating scale for e-cigarettes and a yes/no response scale for SLT. The two types of direct measures may not be equally valid, and differences such as the number and nature of response options are an area of interest for future measurement validation research. Despite its limitations, this analysis provides a model that can and should be replicated with finer-grained scales and cleaner comparisons between direct and indirect measures.

Strengths of this analysis included the use of multiple measures of criterion validity and two product types. The use of national data supports the generalizability of results.

4.1. Conclusion

Beliefs about relative harm are central to research and regulation concerning tobacco and nicotine products, but techniques for measuring such beliefs differ widely. Prior studies suggested that direct measures of perceived relative harm may underestimate the extent to which U.S. adults believe e-cigarettes and SLT are less harmful than cigarettes (20, 21). However, in this first ever validity analysis, direct measures appeared to provide a more valid representation of people's harm beliefs than the indirect measures. Further validation studies will benefit public health research and product regulation.

Highlights.

In tobacco research and regulation, measures of perceived relative harm of products vary.

On direct measures, people compare the harms of products (e.g., e-cigarettes vs. cigarettes).

On indirect measures, people rate the harms of products separately, and ratings are compared.

Direct measures had higher validity than indirect measures, based on product use associations.

Tobacco research and regulation would benefit from more perceived harm measure validation work.

Acknowledgments

Role of Funding Sources

Publication of this article was supported by the U.S. Food and Drug Administration, Center for Tobacco Products, and the U.S. National Cancer Institute, National Institutes of Health. No funding was provided specifically for conducting the analysis, drafting the manuscript, or submitting this paper for publication.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Disclaimer: This publication represents the views of the authors and does not represent FDA/CTP position or policy.

Conflict of Interest Statement: The authors have no conflicts of interest to report.

Contributors

All authors participated in conceptualizing the analysis. AP conducted the statistical analysis. All authors participated in interpreting the results. AP wrote a preliminary draft of the manuscript, and all authors revised the manuscript for critical intellectual content and approved the final version submitted.

1

Only 20 participants (0.70%) rated e-cigarettes as more harmful than cigarettes on the indirect measure. Thus, participants rating e-cigarettes as equally or more harmful than cigarettes were combined into a single category.

2

Only 32 participants (0.72%) rated SLT as more harmful than cigarettes on the indirect measure. Thus, participants rating SLT as equally or more harmful than cigarettes were combined into a single category.

3

The analysis did not examine e-cigarette use status and ever-trying SLT because these variables were not available in HINTS-FDA 2015.

4

We did not examine associations between SLT relative harm measures and ‘ever-trying e-cigarettes,’ or between e-cigarette relative harm measures and ‘ever-trying snus’ or SLT use status.

5

Although not the focus of this analysis, we also calculated these proportions among current cigarette smokers, defined as people who smoked at least 100 cigarettes in their lifetime and currently smoked “some days” or “every day.” 36.9% of current smokers rated e-cigarettes as less harmful than cigarettes on the direct measure, compared to 58.2% on the indirect measure.

6

Among current cigarette smokers, 11.2% rated SLT as less harmful than cigarettes on the direct measure, whereas 34.3% did so on the indirect measure.

References

  • 1.Janz NK, Becker MH. The Health Belief Model: A decade later. Health Education Quarterly. 1984;11(1):1–47. doi: 10.1177/109019818401100101. [DOI] [PubMed] [Google Scholar]
  • 2.Weinstein ND. The precaution adoption process. Health Psychol. 1988;7(4):355–86. doi: 10.1037//0278-6133.7.4.355. [DOI] [PubMed] [Google Scholar]
  • 3.Glanz K, Rimer BK, Viswanath K, editors. Health Behavior and Health Education: Theory, Research, and Practice. 4 ed. Jossey-Bass; San Francisco, CA: 2008. [Google Scholar]
  • 4.Brewer NT, Chapman GB, Gibbons FX, Gerrard M, McCaul KD, Weinstein ND. Meta-analysis of the relationship between risk perception and health behavior: the example of vaccination. Health Psychol. 2007;26(2):136–45. doi: 10.1037/0278-6133.26.2.136. [DOI] [PubMed] [Google Scholar]
  • 5.Pepper JK, Brewer NT. Electronic nicotine delivery system (electronic cigarette) awareness, use, reactions and beliefs: a systematic review. Tob Control. 2014;23(5):375–84. doi: 10.1136/tobaccocontrol-2013-051122. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Borland R, Cooper J, McNeill A, O'Connor R, Cummings KM. Trends in beliefs about the harmfulness and use of stop-smoking medications and smokeless tobacco products among cigarettes smokers: Findings from the ITC four-country survey. Harm Reduct J. 2011;8:21. doi: 10.1186/1477-7517-8-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Biener L, Nyman AL, Stepanov I, Hatsukami D. Public education about the relative harm of tobacco products: an intervention for tobacco control professionals. Tob Control. 2014;23(5):385–8. doi: 10.1136/tobaccocontrol-2012-050814. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Kiviniemi MT, Kozlowski LT. Deficiencies in public understanding about tobacco harm reduction: results from a United States national survey. Harm Reduct J. 2015;12:21. doi: 10.1186/s12954-015-0055-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Song AV, Morrell HE, Cornell JL, Ramos ME, Biehl M, Kropp RY, et al. Perceptions of smoking-related risks and benefits as predictors of adolescent smoking initiation. Am J Public Health. 2009;99(3):487–92. doi: 10.2105/AJPH.2008.137679. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Hughes JR. Harm-reduction approaches to smoking. The need for data. Am J Prev Med. 1998;15(1):78–9. doi: 10.1016/s0749-3797(98)00033-6. [DOI] [PubMed] [Google Scholar]
  • 11.Kozlowski LT, Goldberg ME, Yost BA, White EL, Sweeney CT, Pillitteri JL. Smokers' misperceptions of light and ultra-light cigarettes may keep them smoking. Am J Prev Med. 1998;15(1):9–16. doi: 10.1016/s0749-3797(98)00004-x. [DOI] [PubMed] [Google Scholar]
  • 12.Slovic P. Rejoinder: the perils of Viscusi's analyses of smoking risk perceptions. Journal of Behavioral Decision Making. 2000;13(2):273–6. [Google Scholar]
  • 13.Kozlowski LT, Goldberg ME, Yost BA. Measuring smokers' perceptions of the health risks from smoking light cigarettes. Am J Public Health. 2000;90(8):1318–9. doi: 10.2105/ajph.90.8.1318. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Chapman S, Liberman J. Ensuring smokers are adequately informed: reflections on consumer rights, manufacturer responsibilities, and policy implications. Tob Control. 2005;14(Suppl 2):ii8–13. doi: 10.1136/tc.2005.012591. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Weinstein ND. Accuracy of smokers' risk perceptions. Ann Behav Med. 1998;20(2):135–40. doi: 10.1007/BF02884459. [DOI] [PubMed] [Google Scholar]
  • 16.Choi K, Forster J. Characteristics associated with awareness, perceptions, and use of electronic nicotine delivery systems among young US Midwestern adults. Am J Public Health. 2013;103(3):556–61. doi: 10.2105/AJPH.2012.300947. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Pearson JL, Richardson A, Niaura RS, Vallone DM, Abrams DB. e-Cigarette awareness, use, and harm perceptions in US adults. Am J Public Health. 2012;102(9):1758–66. doi: 10.2105/AJPH.2011.300526. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Pepper JK, Emery SL, Ribisl KM, Rini CM, Brewer NT. How risky is it to use e-cigarettes? Smokers' beliefs about their health risks from using novel and traditional tobacco products. J Behav Med. 2015;38(2):318–26. doi: 10.1007/s10865-014-9605-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Hamilton WL, Norton G, Ouellette TK, Rhodes WM, Kling R, Connolly GN. Smokers' responses to advertisements for regular and light cigarettes and potential reduced-exposure tobacco products. Nicotine Tob Res. 2004;6(Suppl 3):S353–62. doi: 10.1080/14622200412331320752. [DOI] [PubMed] [Google Scholar]
  • 20.Popova L, Ling PM. Perceptions of relative risk of snus and cigarettes among US smokers. Am J Public Health. 2013;103(11):e21–3. doi: 10.2105/AJPH.2013.301547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Wackowski OA, Bover Manderski MT, Delnevo CD. Comparison of Direct and Indirect Measures of E-cigarette Risk Perceptions. Tob Regul Sci. 2016;2(1):38–43. doi: 10.18001/TRS.2.1.4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Kaufman AR, Suls JM, Klein WM. Communicating tobacco product harm: Compared to what? Addict Behav. 2016;52:123–5. doi: 10.1016/j.addbeh.2015.06.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychol Bull. 1955;52(4):281–302. doi: 10.1037/h0040957. [DOI] [PubMed] [Google Scholar]
  • 24.Westen D, Rosenthal R. Quantifying construct validity: two simple measures. J Pers Soc Psychol. 2003;84(3):608–18. doi: 10.1037//0022-3514.84.3.608. [DOI] [PubMed] [Google Scholar]
  • 25.Etter JF, Duc TV, Perneger TV. Validity of the Fagerstrom test for nicotine dependence and of the Heaviness of Smoking Index among relatively light smokers. Addiction. 1999;94(2):269–81. doi: 10.1046/j.1360-0443.1999.94226910.x. [DOI] [PubMed] [Google Scholar]
  • 26.Brewer NT, Weinstein ND, Cuite CL, Herrington JE. Risk perceptions and their relation to risk behavior. Ann Behav Med. 2004;27(2):125–30. doi: 10.1207/s15324796abm2702_7. [DOI] [PubMed] [Google Scholar]
  • 27.Brose LS, Brown J, Hitchman SC, McNeill A. Perceived relative harm of electronic cigarettes over time and impact on subsequent use. A survey with 1-year and 2-year follow-ups. Drug Alcohol Depend. 2015;157:106–11. doi: 10.1016/j.drugalcdep.2015.10.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Morrell HE, Song AV, Halpern-Felsher BL. Predicting adolescent perceptions of the risks and benefits of cigarette smoking: a longitudinal investigation. Health Psychol. 2010;29(6):610–7. doi: 10.1037/a0021237. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Amrock SM, Zakhar J, Zhou S, Weitzman M. Perception of e-cigarette harm and its correlation with use among U.S. adolescents. Nicotine Tob Res. 2015;17(3):330–6. doi: 10.1093/ntr/ntu156. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Wackowski OA, Delnevo CD. Young Adults' Risk Perceptions of Various Tobacco Products Relative to Cigarettes: Results From the National Young Adult Health Survey. Health Educ Behav. 2015 doi: 10.1177/1090198115599988. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Kropp RY, Halpern-Felsher BL. Adolescents' beliefs about the risks involved in smoking “light” cigarettes. Pediatrics. 2004;114(4):e445–51. doi: 10.1542/peds.2004-0893. [DOI] [PubMed] [Google Scholar]
  • 32.Westat . Health Information National Trends Survey 4 (HINTS 4): HINTS-FDA Methodology Report. Rockville, MD: p. 2015. [Google Scholar]
  • 33.Blake KD, Portnoy DB, Kaufman AR, Lin CJ, Lo SC, Backlund E, et al. Rationale, Procedures, and Response Rates for the 2015 Administration of NCI's Health Information National Trends Survey: HINTS-FDA 2015. J Health Commun. 2016;21(12):1269–75. doi: 10.1080/10810730.2016.1242672. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Mazurek JM, Syamlal G, King BA, Castellan RM. Smokeless tobacco use among working adults - United States, 2005 and 2010. MMWR Morb Mortal Wkly Rep. 2014;63(22):477–82. [PMC free article] [PubMed] [Google Scholar]
  • 35.Reyna VF. A theory of medical decision making and health: fuzzy trace theory. Med Decis Making. 2008;28(6):850–65. doi: 10.1177/0272989X08327066. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Chambers JR, Windschitl PD. Biases in social comparative judgments: the role of nonmotivated factors in above-average and comparative-optimism effects. Psychol Bull. 2004;130(5):813–38. doi: 10.1037/0033-2909.130.5.813. [DOI] [PubMed] [Google Scholar]
  • 37.Rose JP. Are direct or indirect measures of comparative risk better predictors of concern and behavioural intentions? Psychol Health. 2010;25(2):149–65. doi: 10.1080/08870440802340164. [DOI] [PubMed] [Google Scholar]
  • 38.Krosnick JA, Presser S. Question and questionnaire design. In: Marsden PV, Wright JD, editors. Handbook of Survey Research. 2 ed. Emerald Group Publishing Limited; Bingley, UK: 2010. pp. 263–313. [Google Scholar]
  • 39.Slovic P. Perception of risk. Science. 1987;236(4799):280–5. doi: 10.1126/science.3563507. [DOI] [PubMed] [Google Scholar]
  • 40.Weaver SR, Pechacek T, Halpern-Felsher BL, Majeed B, Slovic P, editors. Society for Research on Nicotine and Tobacco. Chicago, IL, USA: 2016. Measuring risk perceptions of ENDS from a behavioral risk perception perspective: Lessons learned, current challenges, and implications for research and policy. [Google Scholar]
  • 41.Tan AS, Bigman CA. E-cigarette awareness and perceived harmfulness: prevalence and associations with smoking-cessation outcomes. Am J Prev Med. 2014;47(2):141–9. doi: 10.1016/j.amepre.2014.02.011. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES