Developing and Validating Measures of Absolute and Relative E-Cigarette Product Risk Perceptions: Single Items Can Be Surprisingly Comprehensive

Abstract

Introduction

Tobacco risk perceptions are important predictors of behavior and are impacted by tobacco communications. Our systematic literature review (completed in 2018) found there were no measures of e-cigarette risk perceptions that were completely consistent with tobacco researcher recommendations (eg, specifying use frequency) and had demonstrated validity and reliability. The current study develops measures to assess specific risk perceptions, including absolute risks and risks compared with cigarettes, nicotine replacement therapy, and all nicotine cessation.

Methods and Results

We generated a list of tobacco health effects based on our previous systematic review of tobacco risk perception measures. Based on health effects prioritized by regulatory science experts, we developed 63 items to assess seven types of e-cigarette risk perceptions: absolute health and addiction risks, health and addiction risks relative to cigarettes, pregnancy health risks relative to cigarettes, health risks relative to nicotine replacement therapy, and health risks relative to all nicotine cessation. We fielded these items in an online survey (N = 1642). Through reliability and validity analyses, we reduced this pool to 21 items, including many single-item measures. Supporting the measures’ validity, each measure was negatively associated with current e-cigarette use, e-cigarette intentions, and skepticism about e-cigarette harms; and positively associated with perceiving e-cigarettes as equally or more harmful than cigarettes and intentions to quit e-cigarettes.

Discussion

This study developed and validated brief measures of several types of e-cigarette risk perceptions. Surprisingly, we found that for many types of risk perceptions, multi-item measures were redundant and these perceptions were well-represented by single-item measures.

Implications

This study developed measures of seven types of e-cigarette health risk perceptions, including absolute health and addiction risk, and risk relative to cigarettes, nicotine replacement therapy, and cessation. We reduced 63 items to 21 to measure all of these constructs. These measures follow tobacco researcher recommendations, were developed using a rigorous measures development process, and demonstrated some aspects of reliability and validity. Because these measures are publicly available, they can be used by public health and industry researchers.

Introduction

Researchers, clinicians, and public health groups often seek to measure people’s perceptions of nicotine and tobacco products. Risk perceptions are theoretical determinants of health behavior,^1–3 and across a range of tobacco products, can affect initiation, cessation, and switching.^4–10 For example, researchers have argued that increases in the public’s perceptions of e-cigarette health risks may dissuade young people from initiating e-cigarette use, deter smokers from switching to e-cigarettes, and encourage relapse to cigarette smoking among smokers who had switched to e-cigarettes.^11–13 Additionally, these perceptions can be examined to understand the impact of communications about nicotine and tobacco products (eg, advertising,¹⁴ educational campaigns,¹⁵ news articles¹⁶) and product features (eg, filters¹⁷).

Tobacco risk perceptions are also of interest to regulators. In 2009, the Family Smoking Prevention and Tobacco Control Act gave the U.S. Food and Drug Administration (FDA) broad authority to regulate the manufacture, marketing, sale, and distribution of tobacco products. When companies apply to market specific tobacco products in the United States, the FDA recommends that the companies submit research that includes studies of consumers’ risk perceptions of those specific products.^18,19 Risk perceptions of specific products are associated with intentions to use those products¹⁴ and current use of the products²⁰; thus, they can help FDA evaluate the potential effects of marketing products on population health.

E-cigarettes are the most commonly used tobacco product among youth²¹ and are deemed as tobacco products by the FDA because they contain nicotine derived from tobacco.²² Four types of e-cigarette risk perceptions can be particularly informative to FDA’s regulatory decision-making^18,19: (1) absolute health and addiction risks; (2) health and addiction risks compared with combustible cigarettes; (3) health risks compared with all nicotine cessation, and (4) perceptions of the product relative to nicotine replacement therapy (NRT), a type of FDA-approved cigarette smoking cessation medication. These perceptions are relevant to FDA’s evaluation of consumer understanding of the risks of using particular e-cigarettes and the effect of marketing them on population health.^18,19 For example, these perceptions can help FDA evaluate whether consumers will initiate e-cigarettes use, switch from cigarettes to e-cigarettes, or use e-cigarettes instead of using NRTs or quitting tobacco and nicotine use in applications seeking authorization to market new e-cigarettes or market them with modified risk claims. Additionally, data on these risk perceptions could inform FDA’s evaluation of whether people understand modified risk claims.

The Current Study

We sought to design e-cigarette risk perception measures that are reliable, valid, consistent with researcher recommendations, include a range of health effects, are as brief as possible, and include the seven types of perceptions of relevance to tobacco product regulators^18,19: perceptions of (1–2) absolute health and addiction risks, (3–5) health, addiction, and pregnancy risks compared with cigarettes, (6) risks compared with NRTs, and (7) risks compared with all nicotine cessation.

To accomplish this, we used a literature review and regulatory science expert input to develop a comprehensive pool of items formatted to be consistent with researcher recommendations. Then, we studied the items’ factor structures, narrowed down the item list to create measures, and assessed the measures’ reliability and validity by analyzing inter-item correlations and associations with important constructs. If valid, we expected risk perception measures to be positively associated with general harm perceptions of e-cigarettes compared with cigarettes, and negatively associated with skepticism about the harms of e-cigarettes. Based on previous studies, we expected the risk perception measures to be negatively associated with intentions to use e-cigarettes and current e-cigarette use³⁸; and positively associated with intentions to quit e-cigarette use (among current users).²³ Additionally, based on prior research,¹⁴ we expected perceptions of the health risks of e-cigarettes compared with cigarettes to be lower for participants assigned to view e-cigarette product packages or ads with modified risk information stating that the e-cigarette presents lower risk for cancer than cigarettes.

Methods

Scale Development Overview

We sought to follow a rigorous development process involving: devising a list of items thought to tap into all constructs, narrowing down the list of items based on expert feedback, and assessing reliability and validity.^24,25 Furthermore, we sought to ensure that our measures would be consistent with recommendations from risk perception researchers,^26–28 including:

• writing items in first-person (to account for unrealistic optimism about personal risks²⁹);

• specifying conditions of product use (so participants are clear about the behavior they are rating)^30–32;

• asking about multiple health effects, as multi-item scales can have greater reliability than individual items^26,38;

• using Likert-type rather than numeric scales (they are simpler for participants with innumeracy³³ and can have better predictive validity^34,35); and

• for relative risk perceptions, asking participants to compare products directly (rather than deriving comparisons from absolute ratings of each product) as this may be more predictive of behavior.^36,37

A recent literature review of multi-item measures of tobacco risk perceptions found few e-cigarette measures.³⁸ However, a proprietary measure developed by Philip Morris International to assess perceptions of a heated tobacco product³⁹ addresses many of the above recommendations but still has shortcomings: it does not ask participants to compare e-cigarettes to other products, does not specify frequency of use, and is long (72 items to assess risk perceptions of cigarettes, an e-cigarette or “reduced risk product,” NRTs, and cessation; 36 items in the short version).⁴⁰ In addition, to enable product comparisons, it uses the same set of items to assess perceived risks of different tobacco and nicotine products and averages them together, even though the same health effects might not be relevant or interrelated in the same way for all products.

Item Development

Appendix A summarizes the item-development process. First, we reviewed the literature up to May 2018 to identify multi-item measures of perceptions of tobacco health effects.³⁸ We listed the 70 e-cigarette health effects mentioned in the measures, grouped into six categories: general health harm, addiction, cancer, cardiovascular, pulmonary, and oral. We asked 12 tobacco regulatory science experts at FDA’s Center for Tobacco Products (including a physician, an epidemiologist, a pharmacologist, and social scientists) to identify the health effects that best represented each category. We selected a core set of the seven most frequently identified effects: general effects (having a shorter life, getting sick often, getting a life-threatening disease), addiction, cancer, cardiovascular (having heart problems), pulmonary (having breathing problems), and oral (teeth damage). We also selected an expanded set of 26 frequently identified additional effects (eg, stomach ulcers, pancreatic cancer), plus five pregnancy-related effects (harm to baby, miscarriage, premature birth, low birthweight, birth defect).

We then formatted the health effects into survey items. For the core health effects, we created items assessing each type of e-cigarette risk perception: (1) absolute risk, (2) risk compared with using cigarettes, (3) risk compared with using NRTs, and (4) risk compared with nicotine cessation. For the expanded health effects, we created items assessing risk compared with using cigarettes, to measure these perceptions with additional specificity. We formatted all items based on researcher recommendations (ie, using first person, describing conditions of use, use Likert-type response scales, using direct measures of relative risk perceptions).

Because asking participants to make several comparisons to measure each type of risk perception was complex, we wrote brief instructions to precede each set of items, which included large images of the e-cigarette product and comparison product (cigarettes, NRTs, or a “no tobacco” symbol). After these instructions, a question stem and a list of health effects appear. An example question stem for risk relative to cigarettes was “If you either used Blu e-cigarettes OR cigarettes every day, which product would make it more likely that you would…” Response categories ranged from 1 (MUCH more likely with Blu e-cigarettes) to 5 (MUCH more likely with cigarettes). Small images of each product appeared above each end of the response scale. For pregnancy-related risk items, instead of writing them in first-person, we asked participants to imagine a pregnant woman. We chose not to include a “don’t know” option because we did not want to exclude participants from psychometric scale creation and validation analyses, particularly given that “don’t know” responding on risk perception measures is related to demographics.⁴¹ However, participants who left questions unanswered and attempted to advance to the next page had the answer option “prefer not to answer” appear; they could select this and then advance to the next page.

We cognitively tested items and instructions through eight interviews with cigarette smokers, six of whom also used e-cigarettes. Feedback about the clarity of items and ease of responding was generally positive, and participants particularly liked the images of the products being compared. We addressed several opportunities for improvement: simplifying the instructions; reducing image sizes; briefly describing emphysema; shortening required stimuli viewing time from 15 to 10 seconds; showing the stimuli a second time; and adding numbers below the Likert-type response options. We added items on the risk of getting two specific types of cancer (mouth and lung) to measures of absolute risk and risk relative to cigarettes, and added harm your overall health for participants to rate with each main set of health effects. Appendix B depicts the final instructions and items.

Scale Creation and Validation

Design and Measures

We fielded items in a study with a 3 (brand: Blu, VUSE, or MarkTen) × 2 (package vs. advertisement) × 2 (modified risk claim: present vs. control) between-subjects design conducted October through December 2019. Figure 1 summarizes the survey flow, measures, and hypothesized relationships with risk perceptions. Participants first responded to items measuring: tobacco use, including past 30-day e-cigarette use⁴²; intention to quit e-cigarette use (for every day or someday e-cigarette users, adapted from cigarettes to e-cigarettes⁴³); perceived harm of using e-cigarettes compared with cigarettes⁴⁴; and skepticism of the harms of using e-cigarettes (three items averaged, adapted from cigarettes to e-cigarettes⁴⁵; Cronbach’s alpha = .83). Next, participants viewed the e-cigarette package or advertisement and completed measures of absolute health and addiction risks. We included three brands as a way of enhancing the study’s generalizability and reducing the likelihood that findings would only be applicable to one specific e-cigarette brand’s marketing materials. Participants viewed the stimuli a second time and completed measures of perceived health and addiction risks relative to cigarettes, health risks relative to NRT, and health risks relative to cessation; and a measure of intention to use the product⁴⁶ (four items averaged, Cronbach’s alpha = .94). At the end of the study, participants completed an item assessing recall of the specific modified risk statement (if any) that appeared on the stimuli they viewed.

Figure 1.

Study overview: survey flow and example items, e-cigarette risk perception (RP) validity hypotheses, and whether hypotheses were supported.

Sample

Adults (ages ≥18 years; N = 3278) were recruited from the Lightspeed online survey panel (Table 1). We overrecruited tobacco users, such that approximately equal portions of the sample were tobacco nonusers, past 30-day cigarette smokers, and past-30-day e-cigarette users. Young adults (age 18–25) were overrecruited (39.5% of the sample) because of the larger study’s goals. Because people from online panels are typically more educated than the general population,⁴⁷ we overrecruited people who had a high school diploma, GED, or less.⁴⁸ Except for analyses of how experimental condition affected these constructs, we limited all analyses to the control condition (N = 1642 of the total 3287 total), to avoid effects of the modified risk information.

Table 1.

Sample Characteristics

Variable	N  a	Percent
Age
18–24	659	40.1
25–34	297	18.1
35–44	175	10.7
45–54	152	9.3
55–64	185	11.3
65+	174	10.6
Sex
Male	557	33.9
Female	1074	65.4
Sexual orientation
Straight	1368	83.3
Lesbian or gay	68	4.1
Bisexual	169	10.3
Queer	9	0.5
Something else	26	1.6
Race/ethnicity
White, non-Hispanic	1120	68.2
Black, non-Hispanic	255	15.5
Asian, non-Hispanic	60	3.7
Other or multiple race, non-Hispanic	39	2.4
Hispanic	166	10.1
Education
High school or less	622	37.9
Some college	469	28.6
Bachelor’s or Associate’s degree	466	28.4
Master’s degree or higher	84	5.1
Children in household
No children	1147	69.9
1 or more children	485	30.1
Military service
Not currently serving	1596	97.2
Currently serving	46	2.8
Total	1642	100.0

^aBecause of item missingness, the total N across response categories for some variables does not equal the total sample size.

Stimuli

Stimuli were images of advertisements and packages of e-cigarettes (Appendix C) based on real ads (from trinketsandtrash.com) and packages purchased in 2016. We used three brands (Blu, MarkTen, and VUSE; market leaders in 2016) to reduce the likelihood that results would be attributable to one brand. We edited the text to create versions with and without a modified risk claim (“If cigarette smokers switch completely to this product, they can lower their risk of cancer”), to include ad and package copy about switching completely from cigarettes to the product, and to bear the required nicotine warning (not required at the time of stimuli development).

Analyses

We conducted six principal-axis factor analyses (with oblimin rotation, allowing factors to correlate) to analyze items representing six constructs: absolute health risks, health risks relative to cigarettes, addiction risks relative to cigarettes, pregnancy health risks relative to cigarettes, health risks relative to NRT, and health risks relative to cessation. Factor analyses excluded the item intended to serve as a single-item measure of overall risk perception (“harm your overall health”), and assumed a continuous underlying distribution.

Next, we conducted reliability analyses on items representing each factor. A Cronbach’s alpha of at least .70 reflected adequate internal consistency. A Cronbach’s alpha greater than .90 indicated redundancy between items and could be a basis for further reduction.²⁵ In such cases, we evaluated whether the scale could be well-represented by the “harm your overall health” by examining the correlation between this item and the average score on the scale. A correlation of at least .80 indicated that the overall harm item could represent the risk perception construct and accounted for a majority of shared variance (ie, at least 64%). Otherwise, we retained the eight-core items to represent the risk perception construct.

To evaluate validity, we conducted separate mixed model regression analyses using the seven risk perception measures to predict each criterion variable. We used logistic regression to analyze binary criterion variables (current e-cigarette use, belief that e-cigarettes are less harmful than cigarettes) and linear regression to analyze other criterion variables (intention to quit e-cigarette use, skepticism about e-cigarette harms, and intention to use the e-cigarette product). In each mixed model, risk perception was a fixed effect and brand (Blu, MarkTen, and VUSE) was a random effect. All analyses excluded participants who selected “prefer not to answer” for any item in the scale (about 0.1%–0.4% of the sample for any given item).

Results

Factor Analyses and Item Reduction

For each of the six multi-item risk perception constructs, factor analysis preferred a single-factor solution, with the factor explaining a large proportion (47.4%–91.5%) of the shared variance and most factor loadings ranging between .61 and .99 (Table 2). Cronbach’s alphas were between .70 and .90 for two constructs (addiction risk compared with cigarettes and pregnancy health risk); thus, we retained all corresponding items (Table 2). For the other four constructs (absolute health risks, health risks relative to cigarettes, health risks relative to NRT, and health risks relative to cessation), Cronbach’s alphas were greater than .90, indicating potential redundancy. For three of these constructs (absolute health risks, health risks relative to NRT, and health risks relative to cessation), the correlation between the scale mean and the overall item was at least .80; therefore, we used the overall item to represent these constructs. For health risks compared with cigarettes, this correlation was .75 (under the .80 threshold). Thus, we assessed whether the core eight items well-represented this scale by correlating the average of these items with the average of the 23 remaining scale items; this correlation was .93. Although Cronbach’s alpha was slightly over .90 for this 8-item scale (.91), we decided against arbitrarily eliminating additional items.

Table 2.

Summary of Factor Analysis and Item Reduction Results for Developing the Final Measures of Six Types of E-Cigarette Risk Perceptions

Construct	Initial # items	Factor analysis resultsa	Initial alpha	Final scale description and key statistics	Final # items
Absolute health riskb	8	# factors: 1 % var. explained: 71.6 Factor loadings: .76–.86 Communalities: .57–.74	.94	Used overall harm item to represent scale; correlation between this and all items = .82.	1
Risk of e-cigarette product compared with cigarettesb
Health risk	31	# factors: 1 % var. explained: 47.4 Factor loadings: .68–.75 (except common cold or flu, pancreatic cancer, diabetes, stomach ulcers, stomach cancer <.3). Communalities: .46–.62	.97c	The overall harm item had a correlation with the overall scale of .75, under our threshold. Therefore, we used the core 8-item scale to represent the construct. The average of these items has a .93 correlation with the 23 remaining items.	8
Addiction risk	4	# factors: 1 % var. explained: 66.7 Factor loadings: .72–.79 Communalities: .52–.58	.83	Used all 4 items.	4
Pregnancy risk	5	# factors: 1 % var. explained: 70.4 Factor loadings: .61–.66 Communalities: .55–.62	.90	Used all 5 items.	5
Risk of e-cigarette product compared with cessation and NRT
Health risk relative to NRT	7	# factors: 1 % var. explained: 77.2 Factor loadings: .73–.97 Communalities: .56–.72	.94	Used overall harm item to represent scale; correlation between this and all items = .92.	1
Health risk relative to cessation	7	# factors: 1 % var. explained: 91.5 Factor loadings: .81–.99 Communalities: .77–.80	.97	Used overall harm item to represent scale; correlation between this and all items = .98.	1

For all analyses, items were scored such that higher scores indicate that the product was rated as higher risk. The analytic sample was limited to participants in the control condition (n = 1642).

^aPrincipal axis factor analysis with oblimin rotation.

^bAbsolute addiction risk is excluded from this table as it was measured with a single item.

^cCronbach’s alpha for the final scale was .91.

Altogether, we reduced the number of items measuring the six constructs from 62 to 20, as listed in Table 2. In addition, we fielded a single-item measure of perceived absolute addiction risk, yielding a total of seven risk perception measures (Appendix D) for assessment in validity analyses.

Validity

Each risk perception measure demonstrated criterion validity in several ways (Table 3). First, on all measures, greater risk perceptions were associated with lower odds of being a current e-cigarette user (OR range: 0.39, 0.81). Second, all measures were positively associated with overall perceptions of the harm of e-cigarettes compared with cigarettes: greater risk perceptions of e-cigarettes across all measures were associated with greater odds of perceiving e-cigarettes as equally or more harmful than cigarettes in general (OR range: 1.11, 4.92). Third, among current e-cigarette users, all measures except for one (risk relative to cessation) were positively associated with intentions to quit e-cigarette use (B range: .14, .60). Fourth, all measures were negatively associated with skepticism about the harms of e-cigarettes (B range = −.58, −.22). All measures were also negatively associated with intention to use the specific e-cigarette product (B range: −.39, −.19).

Table 3.

Criterion Validity Results for Measures of the Seven Types of E-Cigarette Product Risk Perceptions

	Criterion validity
	Outcomes
	Current e-cigarette usea	Perceiving e-cigarettes as equally or more harmful than cigarettesb	Intention to quit e-cigarette usec	Skepticism of e-cigarette harms	Intention to use specific e-cigarette
	OR (95% CI)		Unstandardized B (95% CI)
Absolute risk
1. Absolute health risk	.66 (.61, .72)	2.17 (1.97, 2.39)	.46 (.35, .57)	−.35 (−.39, −.31)	−.25 (−.29, −.22)
2. Absolute addiction risk	.79 (.73, .86)	1.35 (1.24, 1.47)	.21 (.09, .32)	−.22 (−.26, −.18)	−.19 (−.23, −.16)
Risk of e-cigarette product compared with cigarettes
3. Health risk	.39 (.33, .45)	4.92 (4.09, 5.92)	.60 (.42, .78)	−.58 (−.65, −.52)	−.39 (−.45, −.34)
4. Addiction risk	.58 (.50, .67)	1.90 (1.64, 2.20)	.23 (.05, .41)	−.40 (−.46, −.33)	−.34 (−.39, −.28)
5. Pregnancy risk	.43 (.38, .50)	3.16 (2.69, 3.71)	.50 (.32, .68)	−.46 (−.53, −.40)	−.34 (−.40, −.28)
Risk of e-cigarette product compared with cessation and NRT
6. Health risk relative to NRT	.77 (.70, .86)	1.21 (1.09, 1.34)	.14 (.01, .28)	−.32 (−.37, −.26)	−.24 (−.28, −.19)
7. Health risk relative to cessation	.81 (.74, .88)	1.11 (1.01, 1.21)	.03 (−.09, .15)	−.26 (−.30, −.21)	−.19 (−.23, −.15)

For all analyses, product risk perception items were scored such that higher scores indicate the e-cigarette product was rated as higher risk. For the scales used in validation, higher scores indicate a higher amount of the construct (eg, higher scores indicate higher intention to quit e-cigarette use). The analytic sample was limited to participants in the control condition (n = 1642).

^aResults of binary logistic regression with scale predicting current e-cigarette use, coded as 0 = nonuser, 1 = user.

^bParticipants were asked whether e-cigarettes were less harmful, equally harmful, or more harmful than cigarettes; responses were coded as 0 = e-cigarettes are less harmful, 1 = e-cigarettes are equally or more harmful.

^cAmong participants reporting “every day” or “some day” e-cigarette use.

Based on the full sample, mean perceived health risk compared with cigarettes did not differ based on whether participants viewed modified risk information, which stated that the e-cigarette product presents lower cancer risk compared with cigarettes (B [95% CI] = −0.04 [−0.09, 0.01]). However, among participants who viewed an ad or package with a modified risk statement, only 39.2% could correctly identify the exact statement from a set of four modified risk claims; in the control condition, 63.3% correctly responded that none of these claims appeared. Among participants who correctly responded, the mean perceived health risk relative to cigarettes indeed differed by modified risk claim exposure: on average, participants who viewed the modified risk claim rated products 0.14 points lower (on a 5-point scale) than participants who did not view the claim (B [95% CI] = −0.14 [−0.21, −0.07]).

Discussion

Our recent systematic literature review found no measures of e-cigarette risk perceptions that were consistent with researcher recommendations and had demonstrated validity and reliability.³⁸ Our research helps address this gap by developing measures of seven types of e-cigarette health risk perceptions: (1–2) absolute health and addiction risks, (3–4) health and addiction risks relative to cigarettes, (5) pregnancy health risks relative to cigarettes, (6) health risks relative to NRT, and (7) health risks relative to all nicotine cessation. These measures follow tobacco researcher recommendations were developed using a rigorous measurement-development process, and demonstrated some aspects of reliability and validity. Because these measures are publicly available, they can be used by public health and industry researchers. As we focused on creating quality measures that minimize respondent burden, all seven of these risk perceptions can be measured with 21 items.

Our factor analyses revealed that people do not tend to distinguish between various tobacco-related health effects when rating the potential risks posed by e-cigarettes, because we generally found a single-factor solution that explained the majority of variance in the items. This means that perceptions of various potential health risks (eg, respiratory, cardiovascular, oral) tended to correlate with one another in one large cluster rather than several distinct ones. When measuring risk perceptions in absolute terms, relative to NRT, and relative to cessation, our findings support the use of a single item, because tests of internal consistency found that the multiple items represented by a single factor were so highly correlated that they were largely redundant. While in general including more than one item can help reduce random measurement error,²⁵ this benefit should be balanced against increased respondent burden. For risk perceptions compared with cigarettes, in contrast, we found less redundancy across the multiple items and thus greater potential benefit of including multiple items. Overall, we reduced 63 items comprehensively measuring seven risk perception constructs to 21. This simplifies the measurement of these perceptions by allowing researchers to use brief measures that focus on key health risks.

Study findings help to demonstrate some aspects of the construct validity of our e-cigarette risk perception measures. Each measure was associated with lower odds of current e-cigarette use, lower intentions to use e-cigarettes, and lower skepticism about the harms of e-cigarettes. Greater risk perceptions of e-cigarettes, in general, were associated with greater odds of perceiving e-cigarettes as equally or more harmful than cigarettes and greater intentions to quit e-cigarette use. Furthermore, besides the scale for risk relative to cessation, all measures also demonstrated expected associations with intentions to quit e-cigarette use e-cigarettes among current users. Additionally, our measure of health risk perceptions relative to cigarettes was sensitive to modified risk claims; participants who saw e-cigarettes with a modified risk claim and were able to recall it had lower perceived relative risk perceptions of the product compared with cigarettes. These observed associations support the validity of our measures of perceived e-cigarette health risks.

Limitations and Future Directions

This study has several limitations. We used a convenience sample that is not representative of national estimates; however, this does not affect internal validity. Because the e-cigarette marketplace and e-cigarette health effect research are rapidly changing, perceptions may also change, and these measures’ validity should be reassessed in the future. Our final measures will not necessarily be adaptable to other nicotine and tobacco product types, as people may perceive the risks of different tobacco-related health conditions uniquely for different products. Additionally, we did not include a “don’t know” response option because doing so has resulted in underserved populations being removed from the main analyses⁴¹; however, this means our data include responses from participants who may be uncertain about e-cigarette harms. Studying certainty of e-cigarette perceptions can be a topic for future research.

Future research could further reduce some of our longer measures (eg, eight items for e-cigarette health risk relative to cigarettes) and optimize response options. This could be done using item response theory, which focuses on the item as the unit of analysis and provides granular, item-level statistical information about a measure that often highlights items that are redundant, have little practical significance, and have underutilized response options. This information could be used to further simplify our measures while retaining their utility. Future research could also use this study’s approach, starting with the full list of items, to develop measures of comparable risk perceptions for other nicotine and tobacco products, such as smokeless tobacco, hookah, and cigars. Additionally, future research could use these measures to assess the extent to which changes in specific risk perceptions predict patterns of product use, such as switching from cigarettes to e-cigarettes.

Conclusion

We developed brief measures of seven key types of e-cigarette health risk perceptions that are relevant to FDA’s regulatory decision-making. Our measures address the noted lack of psychometrically valid measures of e-cigarette risk perceptions that follow tobacco researcher recommendations. Specifically, these measures follow tobacco researcher recommendations such as specifying use frequency, writing items in first person, using Likert-type response scales, and assessing relative risk perception directly. Additionally, these measures were developed using a rigorous measurement-development process (including a literature review and expert feedback), and demonstrated some aspects of reliability and validity (eg, positively related to quit intentions, negatively related to use intentions). Surprisingly, we found that for many types of risk perceptions, multi-item measures were redundant and these perceptions were well-represented by single-item measures. Future research could evaluate their psychometric properties in the context of an evolving tobacco product marketplace. Doing so would increase the utility of our measures for tobacco regulatory science.

Funding

This paper is supported with Federal funds from the Center for Tobacco Products, Food and Drug Administration, Department of Health and Human Services, under contract to Research Triangle Institute (contract no. HHSF223201110005B).

Declaration of Interests

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Food and Drug Administration. The authors do not report any conflicts of interest.

Data Availability

Providing the dataset online is not possible for ethical reasons, because it was not included in the consent form that participants completed before participating.