Abstract
Objectives
We used ecological momentary assessment (EMA) to examine young adults’ attitudes towards pro-tobacco messages encountered in real time and their association with intentions to use tobacco.
Methods
Young adults (N = 92, ages 18–29) recorded sightings of marketing or social media related to tobacco in real time via mobile app for 28 days. Participants reported message characteristics, their attitudes towards the message, and intentions to use the depicted product for each submission. We used generalized linear mixed models to examine factors related to attitude towards message and intentions to use tobacco.
Results
Messages depicting e-cigarettes (p < .001) or hookah (p < .05) were associated with significantly more favorable attitudes compared with traditional cigarettes. Positive attitude towards the message was significantly associated with intention to use the depicted product (p < .001). Messages depicting e-cigarettes and hookah were significantly associated with higher intention to use. Message source was not significantly related to attitudes towards the message or product use intentions.
Conclusions
Marketing featuring e-cigarettes and hookah is an important target for future regulation. Given that pro-tobacco and e-cigarette messages are prevalent online, future research should consider the Internet and social media as important venues for counter-marketing and intervention efforts.
Keywords: ecological momentary assessment, tobacco advertising, social media, young adults
Tobacco use remains the leading cause of preventable disease and death in the United States.1 Young adulthood is a time of transition that provides many opportunities for the adoption and progression of tobacco use due to changes in environment, peer groups, and life stressors,2 and it is well established that nearly all cigarette smoking begins before the age of 26.3 Despite promising declines in cigarette and smokeless tobacco use in the past decade, 16.7% of young adults aged 18–24 are current smokers,4 and the popularity of new and emerging nicotine and tobacco products such as hookah and electronic cigarettes is rising.1 Adults aged 18–24 have the highest rates of smokeless tobacco use,1 cigar use,1,5 hookah use,5 and e-cigarette use6 of any age group, and according to 2012 National Adult Tobacco Survey data, they are the only age group in which use of multiple products is more prevalent than exclusive cigarette use.7 Younger adult smokers comprise a critical population for the survival of the tobacco industry as “replacement smokers,”8 and tobacco industry documents reveal that a great deal of marketing effort is spent not only on trying to solidify brand loyalties of current or experimental young adult smokers, but on recruiting new smokers in this age group as well.9
Marketing research demonstrates that attitude towards an ad acts as a mediator between ad exposure and purchase intentions.10 Although positive attitudes about tobacco advertising are known to be strongly associated with interest and tobacco use intentions among youth,11–14 this relationship is less understood among young adults. Research has shown that young adults are more likely to be attracted to cigarette ads and to own tobacco promotional items compared with their older counterparts,15 and that their exposure to tobacco marketing at bars, nightclubs, and college campuses is associated with a higher prevalence of smoking.16 However, much of the research on tobacco marketing and young adults has focused on factors like the ability to name a favorite tobacco ad,17 ownership of tobacco promotional items,15 or recall of exposure rather than attitudes or the cognitive factors involved in their attitude towards a message.18 The few studies that have examined young adults’ attitudes on tobacco or e-cigarette advertising have been conducted in controlled or laboratory settings, with poor ecological validity, where participants are exposed to ads that have been pre-selected by researchers.19,20
Research demonstrates that traditional tobacco and e-cigarette advertising influences both intentions to use and product use.21,22 However, the landscape of tobacco advertising has changed dramatically in the last decade. User-generated pro-tobacco content, especially regarding e-cigarettes, hookah, and cigars,23,24 has been observed on a multitude of social networking sites such as YouTube,25 Instagram,26 and Facebook.27 Content analyses of e-cigarette discussions on Twitter show that user-generated tweets focus on sharing information, personal opinion, and first-person use or intent, all of which tend to have positive sentiments.28 Yet, no studies to date have examined how this user-generated content is perceived, or if it affects intentions to use. Examining this relationship is challenging because exposure to user-generated pro-tobacco content is dependent on factors that vary from individual to individual,29 such as tobacco use status within one’s social network.
Fortunately, measurement strategies, such as ecological momentary assessment (EMA), are well-suited for examining individual level differences in daily experiences. EMA is a method that repeatedly collects real-time data in a participant’s natural environment.30 This method is unique because it accounts for environmental characteristics of the measurement, and because it reduces the retrospective distortion of data.31 EMA has been used in behavioral science for years, including research examining the environmental and psychological antecedents of cigarette smoking32,33 and smoking cessation attempts.34,35 Several pilot studies demonstrate the feasibility of EMA for capturing tobacco and alcohol marketing exposure.36–38 In this study, we utilized EMA to examine young adults’ attitudes towards pro-tobacco messages encountered in real time, how these attitudes differ by individual characteristics, and how these attitudes are associated with intentions to use tobacco.
METHODS
Participants
Our participants were young adults aged 18–29 living in Austin, Texas. Participants were recruited via printed flyers and online ads. We required that participants speak English and own a smartphone capable of accessing the Internet. Overall, 181 participants completed a baseline survey. Of those, 148 completed all 4 weeks of the study, 10 dropped out, and 23 were dropped due to inactivity.
Procedure
Data collection took place between March and June 2015. Study procedures have been described in detail elsewhere;38 briefly, participants provided informed consent and were asked to download and install SurveySwipe by SurveyAnalytics, a free smartphone application. Participants completed a baseline survey to evaluate their demographics, use of social media and Internet, use of tobacco products, and perceptions of tobacco advertising; they were given instructions via e-mail for participating in the 28-day study.
Daily EMA
Tobacco product use and recall of advertisement sightings were evaluated every 24 hours via a 3-item survey which asked participants to (1) recall the number of messages encountered in the past 24 hours, categorized by source of message (industry vs user-generated), and (2) report if they had used any of the following products in the past 24 hours: traditional cigarettes, cigars, cigarillos or little cigars, smokeless tobacco (eg, chewing tobacco, snuff, snus, or dip), e-cigarettes or vape pens, and hookah (check all that apply). A push notification was delivered to participants’ phones each day at 12:00 noon to remind them that a new daily survey was available. Participants who did not complete the daily survey by 9:00am the following morning received a reminder e-mail. Study participation was considered complete after the participant had submitted 28 daily surveys. Participants who were inactive for 7 consecutive days were sent an e-mail reminding them of the study participation requirements as well as contact information for the research team and a link to unsubscribe from the study if they desired. Participants who remained inactive after 14 days, despite the e-mail reminder, were dropped from the study.
Event EMA
Participants were instructed to record all sightings of marketing or social media related to tobacco or electronic nicotine delivery systems seen during the study period in real time. Products of interest were defined as cigarettes, cigars, little cigars, or cigarillos, chewing tobacco, snuff, snus, or dip, hookah, dissolvables (such Camel orbs, sticks, or strips), and e-cigarettes, vape pens, or personal vaporizers. We asked participants to report both industry-sponsored materials (defined as printed ads or flyers, billboards, coupons or promotional-email offers, online ads, and industry-sponsored social media such as an official Facebook page), as well as user-generated materials (defined as Facebook posts, Instagram photos, Vines, or Tweets, and online discussion threads). All encounters were submitted via a repeatable survey in the SurveySwipe application that allowed participants to upload a photograph or screenshot. Because we were interested in exploring which messages participants noticed without any kind of external influence, participants were not prompted or reminded by the mobile app to report any sighted messages.
Incentives and study completion
After participants had completed 28 daily surveys, they were asked to complete a follow-up survey, in which they recalled their advertising exposure and tobacco product use from the past 30 days, and answered some usability questions related to the mobile app and study protocol. Participants received a gift card for every 7 daily surveys completed during the study period. Due to the possibility that participants might alter their message reporting behavior to increase a financial reward, the decision was made not to award incentives based on the submission of event-driven EMAs. A $10 gift card was awarded for completion of the 7 daily surveys in week one, a $15 gift card for week 2, a $25 gift card for week 3, and a $30 gift card for week 4. In addition, we gave 10 randomly selected participants a $150 gift card for participating in the full 4 weeks of the study.
Measures
Message environment
Participants were asked 6 additional questions upon submission of a product message, including whether the message was industry-sponsored or user-generated, the location of the ad (eg, magazine, billboard, inside a bar or restaurant, on the Internet, etc), the specific website or social network if the message was seen online, which type of product was shown, the brand of the product shown (if any), and if the participant regularly used the shown product.
Attitude towards message
A recipient’s affective reaction to an ad has been shown to be a moderator between exposure to an advertisement and purchase intention.10 Attitude towards the message was operationalized by asking participants to rate on a 5-point Likert scale if they thought the ad was persuasive, interesting, and attractive,39 if they would be likely to try the product based on the message, and if they liked the message overall.40 Higher scores indicated a more positive attitude. These 5 items were averaged to create a composite measure (M = 2.18, SD = 1.04), and the scale showed acceptable internal consistency (Cronbach’s alpha = 0.921).
Product intentions
Product intentions capture the extent to which an individual envisions themselves as someone willing to use that product. Message-specific intentions to use were operationalized via 5 items asking participants to rate on a 5-point scale from strongly disagree to strongly agree how much they agree with the following statements: (1) I’m curious about it; (2) I can see myself trying it; (3) I can see myself using it on my own; (4) I can see myself using it with friends; and (5) I can see myself buying it.40 Items were averaged to create a composite measure (M = 1.98, SD = 1.08, Cronbach’s alpha = 0.927).
Covariates
Several covariates known to influence smoking and tobacco behaviors were measured, including age,3 sex (1 = male, 0 = female),41 race (coded as a series of dummy variables, including 1 = white, 2 = black or African-American, 3 = Asian, 4 = American Indian or Alaska Native, 5 = Native Hawaiian or Other Pacific Islander, and 6 = other, where 1 = yes, 0 = no), Hispanic ethnicity (where 1 = yes, 0 = no),42 and enrollment in a 2- or a 4-year college/university (1 = yes, 0 = no).43 Additional information regarding covariate measures of sensation seeking and skepticism toward advertising are provided below.
Sensation-seeking
Sensation-seeking, defined as a need for physiological arousal, novel experience, and a willingness to take social, physical, and financial risk to obtain such arousal,44 is a personality trait associated with risky behaviors, including cigarette smoking.45,46 To evaluate sensation seeking, we used the Brief Sensation Seeking Scale-4, developed by Stephenson et al.47 The scale consists of 4 items and asks the participant how much they agree or disagree with the following statements on a 5-point scale from strongly disagree to strongly agree: (1) I would like to explore strange places; (2) I like to do frightening things; (3) I like new and exciting experiences even if I have to break the rules; and (4) I prefer friends who are exciting and unpredictable.47
Skepticism toward advertising
Skepticism toward advertising is defined as the general tendency toward disbelief of advertising claims. Skepticism is hypothesized as a general trait that varies across individuals and is related to general persuadability,48 as well as attitude toward ads, believability of ad claims, and perceived influence of the ads.49 Skepticism toward advertising was operationalized using 3 items taken from Obermiller and Spangenberg’s 18-item scale, asking participants how much they agree or disagree with the following statements: (1) Most advertising tells the truth; (2) Advertising is generally informative; and (3) Advertising is a reliable source of information about the product, on a 5-point scale from strongly disagree to strongly agree.48 Items were averaged to create a mean score (M = 2.17, SD = .75). The scale showed acceptable internal consistency (Cronbach’s alpha = .749).
Data Analysis
To fulfill the study aims, we utilized 2 generalized linear mixed models to examine the outcomes of (1) positive attitude towards a message, and (2) product intentions. In both models, we examined fixed effects for all measured characteristics of the message, including type of product, source of message (industry-sponsored or user-generated), and location of message. Covariates of skepticism toward advertising, sensation-seeking age, sex, race/ethnicity, and baseline tobacco use also were included. We included a random intercept to account for individual differences across participants in both models. In the second model examining product intentions as the outcome, we also included a fixed effect for attitude towards message. We conducted analyses using IBM SPSS Statistics for Windows, Version 22.0.
RESULTS
Sample Characteristics
Ninety-two participants submitted at least one event-driven EMA survey. Table 1 shows participant characteristics.
Table 1.
Characteristic | % or Mean (N or SD) N = 92 |
---|---|
Age | 21.21 (2.9) |
Female | 76.1% (70) |
Hispanic | 16.3% (15) |
Race | |
| |
White | 62.0% (57) |
Black | 5.4% (5) |
Asian | 25.0% (23) |
Other/More than one | 7.6% (7) |
| |
Student | 81.5% (75) |
Tobacco Usea | |
| |
Non-user | 69.6% (64) |
Tobacco only | 15.2% (14) |
E-cigarettes only | 2.2% (2) |
Dualb | 13.0% (12) |
Note.
Tobacco use defined as use on at least one day during the last 30 days
Dual defined as the use of both e-cigarettes and a tobacco product during the last 30 days
Message Characteristics
A total of 358 message encounters were submitted. We limited analyses to pro-tobacco messages, leaving a sample of 322 observations. Participants submitted between 0 and 36 industry-sponsored messages and between 0 and 30 user-generated messages each during the 28-day assessment period. Participants included a photograph or screen-shot of the message in 71.2% of the submissions; those who were unable to take a picture or screen-shot cited reasons such as driving at the time of the message sighting, or feeling uncomfortable taking a picture in public. Most reported messages were industry-sponsored (56.2%), 35.7% were user-generated, and 8.1% of messages were categorized as “other” if the source was unclear or the submission was outside of the scope of the study instructions (eg, a report of an individual smoking in public).
The largest category of messages was seen on the Internet (42.2%), followed by 36.3% at a retail point-of-sale (inside or outside a convenience store, gas station, grocery store, drug store, or smoke shop). Of the messages seen online, most were seen on Instagram (32.4%), Facebook (21.3%), Snapchat (12.5%), and Twitter (9.6%). Traditional cigarettes were the most common message subject (45.7%), followed by e-cigarettes (32.0%), and cigars (8.1%). The type of product was coded as “other” in instances where it was not possible to determine what product was depicted in the message (for example, the message showed smoke but not a product).
Predicting Positive Attitude toward Message
Table 2 presents the results of the generalized linear mixed model predicting attitude toward tobacco and e-cigarette messages. Messages depicting e-cigarettes (p < .001) and hookah (p = .02) were associated with significantly higher attitudes compared to traditional cigarettes. Participants reporting tobacco use (p < .001), e-cigarette use (p = .02), or dual use of both tobacco and e-cigarettes (p <.001) at baseline had significantly higher positive attitudes towards messages compared with non-users. Neither the source of the message (ie, industry-sponsored vs user-generated), message location, skepticism toward advertising, and covariates such as sex, age, and race/ethnicity were significantly related to attitude. However, participants with close friends who used e-cigarettes were significantly less likely to report a positive attitude toward a message compared to those with close friends who used (p = .04).
Table 2.
Model Term | Fixed Coefficients | t | Sig. | 95% CI | |
---|---|---|---|---|---|
| |||||
Coefficient | Std. Error | ||||
Intercept | 1.32 | 0.79 | 1.67 | .10 | (−0.23, 2.88) |
Product Depicted in Messagea | |||||
| |||||
Cigars | 0.13 | 0.20 | 0.67 | .50 | (−0.26, 0.53) |
Smokeless tobacco | −0.26 | 0.24 | −1.11 | .27 | (−0.74, 0.21) |
E-cigarettes | 0.33 | 0.12 | 2.83 | <.001 | (0.10, 0.56) |
Hookah | 0.54 | 0.23 | 2.37 | .02 | (0.09, 0.99) |
Other | −0.11 | 0.33 | −0.34 | .74 | (−0.76, 0.54) |
| |||||
Message Source: User-generatedb | 0.01 | 0.24 | 0.05 | .96 | (−0.46, 0.49) |
Message Locationc | |||||
| |||||
Magazine | 0.38 | 0.31 | 1.20 | .23 | (−0.24, 0.99) |
Billboard | 0.29 | 0.27 | 1.07 | .28 | (−0.24, 0.81) |
On TV or in a movie | 0.24 | 0.25 | 0.99 | .32 | (−0.24, 0.73) |
On the Internet | 0.53 | 0.32 | 1.67 | .10 | (−0.10, 1.15) |
Other | 0.21 | 0.25 | 0.83 | .41 | (−0.29, 0.70) |
| |||||
Female | −0.04 | 0.22 | −0.16 | .87 | (−0.47, 0.40) |
Age | −0.03 | 0.03 | −0.90 | .37 | (−0.09, 0.03) |
Race/Ethnicityd | |||||
| |||||
Black/Other | −0.26 | 0.28 | −0.94 | .35 | (−0.81, 0.29) |
Asian | 0.06 | 0.20 | 0.31 | .76 | (−0.33, 0.45) |
Hispanic | −0.02 | 0.23 | −0.09 | .93 | (−0.47, 0.43) |
Baseline Tobacco Usee | |||||
Tobacco only | 1.05 | 0.25 | 4.26 | <.001 | (0.56, 1.53) |
E-cigarettes only | 1.11 | 0.46 | 2.41 | .02 | (0.20, 2.02) |
Dual user | 1.47 | 0.28 | 5.21 | <.001 | (0.91, 2.02) |
Close Friends Who Usef | |||||
Cigarettes | −0.07 | 0.18 | −0.39 | .70 | (−0.42, 0.28) |
Cigars | 0.06 | 0.18 | 0.30 | .76 | (−0.31, 0.42) |
Smokeless tobacco | 0.11 | 0.20 | 0.53 | .60 | (−0.29, 0.51) |
E-cigarettes | −0.49 | 0.24 | −2.06 | .04 | (−0.96, −0.02) |
Hookah | 0.36 | 0.27 | 1.34 | .18 | (−0.17, 0.88) |
| |||||
Skepticism Toward Advertising | 0.10 | 0.12 | 0.88 | .38 | (−0.13, 0.33) |
Sensation Seeking | 0.15 | 0.12 | 1.24 | .22 | (−0.09, 0.38) |
| |||||
Random Effect Covariance | Random Effect | Z | p | 95% CI | |
| |||||
Estimate | Std. Error | ||||
| |||||
Intercept | 0.17 | 0.08 | 1.98 | .05 | (0.06, 0.45) |
Note.
Cigarettes = reference
Industry-sponsored = reference
Retail POS/bar/restaurant = reference
White =reference
Non-user = reference
No close friends who use = reference
Predicting Product Intentions
Table 3 presents the results of the generalized linear mixed model predicting product intentions from attitudes toward tobacco and e-cigarette messages. Positive attitudes towards messages were significantly associated with intentions to use the product depicted (p < .001). Messages that depicted e-cigarettes, hookah, and messages categorized as “other” (ie, showed smoke but no specific tobacco product) were significantly associated with higher intentions to use compared with messages depicting traditional cigarettes. Participants who reported dual use at baseline were significantly more likely to report higher intentions to use (p < .001) compared with non-users; however, no other covariates were significantly associated with product intentions.
Table 3.
Model Term | Fixed Coefficients | t | Sig. | 95% CI | ||
---|---|---|---|---|---|---|
| ||||||
Coefficient | Std. Error | |||||
Intercept | −0.05 | 0.60 | −0.08 | .94 | (−1.22, 1.13) | |
Attitude towards Message | 0.69 | 0.04 | 18.71 | <.001 | (0.62, 0.77) | |
Product Depicted in Messagea | ||||||
| ||||||
Cigars | −0.05 | 0.12 | −0.41 | .69 | (−0.29, 0.19) | |
Smokeless tobacco | 0.06 | 0.14 | 0.39 | .70 | (−0.23, 0.34) | |
E-cigarettes | 0.29 | 0.07 | 3.93 | <.001 | (0.14, 0.43) | |
Hookah | 1.00 | 0.14 | 7.15 | <.001 | (0.73, 1.28) | |
Other | 0.72 | 0.20 | 3.64 | <.001 | (0.33, 1.11) | |
| ||||||
Message Source: User-generatedb | −0.17 | 0.15 | −1.15 | .25 | (−0.46, 0.12) | |
Message Locationc | ||||||
| ||||||
Magazine | −0.35 | 0.20 | −1.76 | .08 | (−0.75, 0.04) | |
Billboard | −0.11 | 0.17 | −0.68 | .50 | (−0.45, 0.22) | |
On TV or in a movie | −0.33 | 0.21 | −1.59 | .11 | (−0.74, 0.08) | |
On the Internet | −0.06 | 0.16 | −0.41 | .68 | (−0.38, 0.25) | |
Other | 0.05 | 0.15 | 0.34 | .73 | (−0.25, 0.36) | |
| ||||||
Baseline Tobacco Used | ||||||
| ||||||
Tobacco only | 0.18 | 0.19 | 0.95 | .34 | (−0.2, 0.56) | |
E-cigarettes only | 0.35 | 0.37 | 0.94 | .35 | (−0.38, 1.09) | |
Dual user | 0.80 | 0.22 | 3.63 | .00 | (0.36, 1.23) | |
| ||||||
Close Friends Who Usee | ||||||
| ||||||
Cigarettes | 0.04 | 0.14 | 0.27 | .79 | (−0.23, 0.31) | |
Cigars | −0.01 | 0.14 | −0.05 | .96 | (−0.29, 0.27) | |
Smokeless tobacco | 0.21 | 0.16 | 1.37 | .17 | (−0.09, 0.52) | |
E-cigarettes | 0.11 | 0.18 | 0.61 | .54 | (−0.25, 0.47) | |
Hookah | 0.00 | 0.21 | 0.02 | .98 | (−0.40, 0.41) | |
| ||||||
Skepticism Toward Advertising | 0.03 | 0.09 | 0.33 | .74 | (−0.15, 0.21) | |
Sensation Seeking | 0.04 | 0.09 | 0.46 | .65 | (−0.13, 0.21) | |
| ||||||
Random Effect Covariance | Random Effect | Z | p | 95% CI | ||
| ||||||
Estimate | Std. Error | |||||
| ||||||
Intercept | 0.15 | 0.05 | 2.80 | .01 | (0.08, 0.30) |
Note.
Model adjusted for gender, age, and race/ethnicity.
Cigarettes = reference
Industry-sponsored = reference
Retail POS/bar/restaurant = reference
Non-user = reference
No close friends who use = reference
DISCUSSION
We examined the factors associated with positive attitudes towards pro-tobacco and e-cigarette messages and the relationship between positive attitudes and intentions to use. Overall, messages depicting e-cigarettes or hookah were associated with significantly higher positive attitudes compared with traditional cigarettes. Participants who reported use of tobacco and e-cigarettes at baseline had more favorable attitudes toward messages than their non-user counterparts; however, participants who had close friends who use e-cigarettes had significantly less favorable attitudes compared with those who did not. Positive attitude towards a message was significantly associated with intentions to use tobacco and e-cigarettes. In addition, messages that depicted e-cigarettes, hookah, or an ambiguous product (for example, the message depicted smoke, or included paraphernalia that could be used for either tobacco or marijuana), were also associated with higher intentions to use.
The finding that positive attitude and intentions to use were higher for messages about hookah and e-cigarettes is consistent with national trends that demonstrate the popularity of both products among young adults. Adults aged 18–25 have the highest rates of hookah use5 and e-cigarette use6 of any age group, and research suggests that hookah and e-cigarettes are perceived to be less harmful and less addictive by young adults than conventional cigarettes.50 Two similar studies examining attitudes and receptivity to e-cigarette marketing among college students found that more positive attitudes towards e-cigarette advertisements were associated with perceptions that e-cigarettes were less harmful than cigarettes, as well as greater intentions to use e-cigarettes.51,52 Few studies have characterized hookah marketing in-depth, however research has documented the prevalence of hookah-related websites and messages online that feature positive sentiments53 and misinformation related to the health consequences of hookah use.54 For example, a content analysis of tobacco-related videos on YouTube revealed that portrayals of hookah were positive, were less likely to mention potential health effects, and were more likely to describe smoking tricks and techniques compared to cigarette-related videos.55
This study was unique in that our methods allowed for examination of exposure to user-generated pro-tobacco content on social media. Although other studies have conducted content analyses of tobacco and e-cigarette-related marketing on social media,28,56 to our knowledge, this is the first study to examine real-time exposure to user-generated messages directly. We wanted to examine the specific effect of user-generated messages on intentions to use tobacco and e-cigarettes due to the signifi-cant relationship between tobacco use and peer influence.57 Having close friends who use tobacco products is consistently associated with tobacco use among young adults,57–59 and young adult tobacco use is also predicted by the perception that peers’ social normative beliefs are supportive of tobacco use.60,61 Despite this fact, our results demonstrated that user-generated messages were not significantly associated with a more favorable attitude or stronger intentions to use tobacco or e-cigarettes compared with industry-sponsored messages. Whereas user-generated messages accounted for only 35.7% of the total messages submitted, their nature and content were more subtle and varied compared to industry messages, which tended to have explicit messages encouraging the viewer to use the portrayed product.
The current study has several limitations. First, the sample was limited to a relatively small and homogenous group of college students who were primarily white, female, non-users; thus, the generalizability of our results may be limited. Whereas young adults remain an important target for tobacco research, future studies could benefit from a more diverse sample with a greater proportion of tobacco and e-cigarette users. Second, it is unknown to what extent the EMA message submissions described here are representative of the nature and extent of messages actually encountered by participants. Participants were instructed to record all sightings of marketing or social media related to tobacco or electronic nicotine delivery systems seen during the study period on a momentary basis; however, they were not prompted or reminded by the mobile app to do so. We have previously reported38 that correlations between the number of messages reported via the daily survey (that utilized a reminder via mobile push notification) and the event-driven EMA varied widely across participants, and some participants reported forgetting to record messages in the final study survey. Finally, there was some discrepancy regarding the characteristics of messages in the daily survey versus the event-driven EMA survey. Specifically, based on the daily survey data, 62.1% of messages seen were user-generated, and 37.9% were industry-sponsored; to the contrary, the event-driven EMA survey suggested the opposite – that the majority of messages encountered were industry-sponsored. Pearson’s chi-square tests revealed no statistically significant differences by sex, race/ethnicity, student status, or tobacco use status between those who submitted event-driven EMA messages versus those who did not; however, it is possible that the participants who were more active in reporting event-driven EMA encounters had fewer contacts on social media that posted tobacco or e-cigarette related content.
An important and unique strength of this study is the ecological validity of assessing real-time, natural exposure to tobacco and e-cigarette messages. Previous studies either have depended on recognition and recall of advertising, or controlled exposure to messages in a laboratory setting to examine participants’ attitude or receptivity to marketing.20,62 However, we were able to assess immediate attitudes towards messages seen in natural environments, including messages seen online and on social media, which tend to be highly varied and individualized depending on users’ online habits and social networks.63,64 In addition to participants’ subjective descriptions of the messages sighted, we also collected objective data in the form of photographs and screenshots. Although we did not do a content analysis of the submitted images, future research utilizing this methodology could explore factors such as imagery, theme, color, and placement to determine what characteristics of messages affect attitudes and intentions to use tobacco and e-cigarettes among young adults. Furthermore, although this study focused on pro-tobacco messages, the utility of the EMA technique may be useful in evaluating the effectiveness of tobacco counter-marketing campaigns, both in terms of overall reach as well as effectiveness in changing attitudes or intentions towards tobacco products.
IMPLICATIONS FOR TOBACCO REGULATION
Our results suggest that messages depicting e-cigarettes and hookah are associated with more favorable attitude and greater use intentions among young adults. Although the 2016 US Food & Drug Administration (FDA) deeming regulation requires health warnings to be included in any advertising for all tobacco products, the marketing of e-cigarettes and hookah is otherwise unrestricted.65 Given that young adults aged 18–25 have the highest rates of hookah5 and e-cigarette use,6 and that they tend to perceive the use of these products as safe and non-addictive, marketing for these products remain an important target for regulation. E-cigarette advertising frequently includes content that appeals to youth and young adults, such as cartoon characters, celebrity endorsements, and the explicit promotion of flavors.66,67 Furthermore, both e-cigarettes and hookah are widely promoted online and on social media websites with large youth audiences.54,68,69 Research suggests that a majority of e-cigarette content on social media is industry-sponsored.70 Whereas the FDA cannot regulate user-generated social media content, brands often incentivize high-profile users to promote their products, and some studies demonstrate an overlap in content between e-cigarette marketers and user-generated content from “vaper enthusiasts.”71 Thus, initial steps to limit the presence of industry-sponsored e-cigarette content on social media may have a significant impact on its overall online prevalence. In summary, future regulations that restrict the content and venues for e-cigarette and hookah advertising may be an important step in preventing youth initiation of these products.
Acknowledgments
Research reported in this publication was supported by grant number (1 P50 CA180906) from the National Cancer Institute and the FDA Center for Tobacco Products (CTP). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the Food and Drug Administration.
Footnotes
Human Subjects Statement
The Institutional Review Board at The University of Texas Health Science Center at Houston approved this study procedure.
Conflict of Interest Statement
The authors have no conflicts of interest to declare.
Contributor Information
Emily T. Hébert, Postdoctoral Research Fellow, Oklahoma Tobacco Research Center, University of Oklahoma Sciences Center, Oklahoma City, OK.
Elizabeth A. Vandewater, Senior Research Fellow, Population Research Center, University of Texas at Austin, Austin, TX.
Michael S. Businelle, Associate Professor, Oklahoma Tobacco Research Center, University of Oklahoma Sciences Center, Oklahoma City, OK.
Melissa B. Harrell, Associate Professor, The University of Texas Health Science Center at Houston-UTHealth, School of Public Health, Department of Epidemiology, Human Genetics and Environmental Sciences, Austin, TX.
Steven H. Kelder, Professor, Department of Epidemiology, Human Genetics and Environmental Sciences, Austin, TX.
Cheryl L. Perry, Professor, The University of Texas Health Science Center at Houston- UTHealth, School of Public Health, Department of Health Promotion and Behavioral Sciences, Austin, TX.
References
- 1.US Department of Health and Human Services (USD-HHS) The Health Consequences of Smoking—50 Years of Progress. A Report of the Surgeon General. Atlanta, GA: USDHHS, US Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2014. [Google Scholar]
- 2.Ling PM, Glantz SA. Tobacco industry research on smoking cessation: recapturing young adults and other recent quitters. J Gen Intern Med. 2004;19(5 Pt 1):419–426. doi: 10.1111/j.1525-1497.2004.30358.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.US Department of Health and Human Services (USD-HHS) Preventing Tobacco Use Among Youth and Young Adults: A Report of the Surgeon General. Atlanta, GA: US-DHHS, US Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2012. [Google Scholar]
- 4.Jamal A, Homa DM, O’Connor E, et al. Current cigarette smoking among adults–United States, 2005–2014. MMWR Morb Mortal Wkly Rep. 2015;64(44):1233–1240. doi: 10.15585/mmwr.mm6444a2. [DOI] [PubMed] [Google Scholar]
- 5.Johnston LD, O’Malley PM, Miech RA, et al. Monitoring the Future national results on drug use: 1975–2013: overview, key findings on adolescent drug use. Ann Arbor: Institute for Social Research, The University of Michigan; 2014. [Accessed November 2, 2017]. Available at: http://www.monitoringthefuture.org/pubs/monographs/mtf-overview2013.pdf. [Google Scholar]
- 6.Schoenborn CA, Gindi RM. Electronic cigarette use among adults: United States, 2014. NCHS Data Brief. 2015;(217):1–8. [PubMed] [Google Scholar]
- 7.Lee YO, Hebert CJ, Nonnemaker JM, Kim AE. Multiple tobacco product use among adults in the United States: cigarettes, cigars, electronic cigarettes, hookah, smokeless tobacco, and snus. Prev Med. 2014;62:14–19. doi: 10.1016/j.ypmed.2014.01.014. [DOI] [PubMed] [Google Scholar]
- 8.Burrows DS R.J. Reynolds Tobacco Co. Younger Adult Smokers: Strategies and Opportunities. [Accessed November 2, 2017];Marketing to Youth. 1984 Feb 29; Available at: https://industry-documents.library.ucsf.edu/tobacco/docs/hjhj0045.
- 9.Ling PM, Glantz SA. Why and how the tobacco industry sells cigarettes to young adults: evidence from industry documents. Am J Public Health. 2002;92(6):908–916. doi: 10.2105/ajph.92.6.908. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lutz RJ, MacKenzie SB, Belch GE. Attitude toward the ad as a mediator of advertising effectiveness: determinants and consequences. Adv Consum Res. 1983;10(1):532–539. [Google Scholar]
- 11.Arnett JJ, Terhanian G. Adolescents’ responses to cigarette advertisements: links between exposure, liking, and the appeal of smoking. Tob Control. 1998;7(2):129–133. doi: 10.1136/tc.7.2.129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Unger JB, Johnson CA, Rohrbach LA. Recognition and liking of tobacco and alcohol advertisements among adolescents: relationships with susceptibility to substance use. Prev Med. 1995;24(5):461–466. doi: 10.1006/pmed.1995.1074. [DOI] [PubMed] [Google Scholar]
- 13.Hawkins K, Hane AC. Adolescents’ perceptions of print cigarette advertising: a case for counteradvertising. J Health Commun. 2000;5(1):83–96. doi: 10.1080/108107300126786. [DOI] [PubMed] [Google Scholar]
- 14.Pinilla J, Gonzalez B, Barber P, Santana Y. Smoking in young adolescents: an approach with multilevel discrete choice models. J Epidemiol Community Health. 2002;56(3):227–232. doi: 10.1136/jech.56.3.227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Biener L, Albers AB. Young adults: vulnerable new targets of tobacco marketing. Am J Public Health. 2004;94(2):326–330. doi: 10.2105/ajph.94.2.326. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Rigotti NA, Moran SE, Wechsler H. US College students’ exposure to tobacco promotions: prevalence and association with tobacco use. Am J Public Health. 2005;95(1):138–144. doi: 10.2105/AJPH.2003.026054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Song AV, Ling PM, Neilands TB, Glantz SA. Smoking in movies and increased smoking among young adults. Am J Prev Med. 2007;33(5):396–403. doi: 10.1016/j.amepre.2007.07.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Choi K, Forster JL. Frequency and characteristics associated with exposure to tobacco direct mail marketing and its prospective effect on smoking behaviors among young adults from the US Midwest. Am J Public Health. 2014;104(11):2179–2183. doi: 10.2105/AJPH.2014.302123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Smith DM, Bansal-Travers M, O’Connor RJ, et al. Associations between perceptions of e-cigarette advertising and interest in product trial amongst US adult smokers and non-smokers: results from an internet-based pilot survey. Tob Induc Dis. 2015;13(1):14. doi: 10.1186/s12971-015-0039-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Villanti AC, Rath JM, Williams VF, et al. Impact of exposure to electronic cigarette advertising on susceptibility and trial of electronic cigarettes and cigarettes in US young adults: a randomized controlled trial. Nicotine Tob Res. 2015;18(5):1331–1339. doi: 10.1093/ntr/ntv235. [DOI] [PubMed] [Google Scholar]
- 21.Lovato C, Watts A, Stead LF. Impact of tobacco advertising and promotion on increasing adolescent smoking behaviours. Cochrane Database Syst Rev. 2011;10:CD003439. doi: 10.1002/14651858.CD003439.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Farrelly MC, Duke JC, Crankshaw EC, et al. A randomized trial of the effect of e-cigarette TV advertisements on intentions to use e-cigarettes. Am J Prev Med. 2015;49(5):686–693. doi: 10.1016/j.amepre.2015.05.010. [DOI] [PubMed] [Google Scholar]
- 23.Link AR, Cawkwell PB, Shelley DR, Sherman SE. An exploration of online behaviors and social media use among hookah and electronic-cigarette users. Addict Behav Rep. 2015;2:37–40. doi: 10.1016/j.abrep.2015.05.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Emery SL, Vera L, Huang J, Szczypka G. Wanna know about vaping? Patterns of message exposure, seeking and sharing information about e-cigarettes across media platforms. Tob Control. 2014;23(Suppl 3):iii17–iii25. doi: 10.1136/tobaccocontrol-2014-051648. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bromberg JE, Augustson EM, Backinger CL. Portrayal of smokeless tobacco in YouTube videos. Nicotine Tob Res. 2012;14(4):455–462. doi: 10.1093/ntr/ntr235. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Richardson A, Ganz O, Vallone D. The cigar ambassador: how Snoop Dogg uses Instagram to promote tobacco use. Tob Control. 2014;23(1):79–80. doi: 10.1136/tobaccocontrol-2013-051037. [DOI] [PubMed] [Google Scholar]
- 27.Freeman B, Chapman S. British American tobacco on Facebook: undermining article 13 of the global world health organization framework convention on tobacco control. Tob Control. 2010;19(3):e1–e9. doi: 10.1136/tc.2009.032847. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Cole-Lewis H, Pugatch J, Sanders A, et al. Social listening: a content analysis of e-cigarette discussions on Twitter. J Med Internet Res. 2015;17(10):e243. doi: 10.2196/jmir.4969. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Pariser E. The Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Think. New York, NY: Penguin Books; 2011. [Google Scholar]
- 30.Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008;4(1):1–32. doi: 10.1146/annurev.clinpsy.3.022806.091415. [DOI] [PubMed] [Google Scholar]
- 31.Stone AA, Shiffman SS, DeVries MW. Ecological momentary assessment. In: Kahneman D, Diener E, Schwarz N, editors. The Foundations of Hedonic Psychology. New York, NY: Russell Sage Foundation; 1999. pp. 26–39. [Google Scholar]
- 32.Shiffman S, Gwaltney CJ, Balabanis MH, et al. Immediate antecedents of cigarette smoking: an analysis from ecological momentary assessment. J Abnorm Psychol. 2002;111(4):531–545. doi: 10.1037//0021-843x.111.4.531. [DOI] [PubMed] [Google Scholar]
- 33.Businelle MS, Ma P, Kendzor DE, et al. Using intensive longitudinal data collected via mobile phone to detect imminent lapse in smokers undergoing a scheduled quit attempt. J Med Internet Res. 2016;18(10):e275. doi: 10.2196/jmir.6307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Watkins KL, Regan SD, Nguyen N, et al. Advancing cessation research by integrating EMA and geospatial methodologies: associations between tobacco retail outlets and real-time smoking urges during a quit attempt. Nicotine Tob Res. 2014;16(Suppl 2):S93–S101. doi: 10.1093/ntr/ntt135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Businelle MS, Ma P, Kendzor DE, et al. Predicting quit attempts among homeless smokers seeking cessation treatment: an ecological momentary assessment study. Nicotine Tob Res. 2014;16(10):1371–1378. doi: 10.1093/ntr/ntu088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Martino SC, Scharf DM, Setodji CM, Shadel WG. Measuring exposure to protobacco marketing and media: a field study using ecological momentary assessment. Nicotine Tob Res. 2012;14(4):398–406. doi: 10.1093/ntr/ntr223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Scharf DM, Martino SC, Setodji CM, et al. Middle and high school students’ exposure to alcohol- and smoking-related media: a pilot study using ecological momentary assessment. Psychol Addict Behav. 2013;27(4):1201–1206. doi: 10.1037/a0032555. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Hébert ET, Vandewater EA, Businelle MS, et al. Feasibility and reliability of a mobile tool to evaluate exposure to tobacco product marketing and messages using ecological momentary assessment. Addict Behav. 2017;73:105–110. doi: 10.1016/j.addbeh.2017.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Peterson RA, Wilson WR, Brown SP. Effects of advertised customer satisfaction claims on consumer attitudes and purchase intention. J Advert Res. 1992;32(2):34–40. [Google Scholar]
- 40.Holmes JH, Crocker KE. Predispositions and the comparative effectiveness of rational, emotional and discrepant appeals for both high involvement and low involvement products. J Acad Mark Sci. 1987;15(1):27–35. [Google Scholar]
- 41.Grunberg NE, Winders SE, Wewers ME. Gender differences in tobacco use. Health Psychol. 1991;10(2):143–153. [PubMed] [Google Scholar]
- 42.US Department of Health and Human Services (USD-HHS) Tobacco Use Among U.S. Racial/Ethnic Minority Groups - African Americans, American Indians and Alaska Natives, Asian Americans and Pacific Islanders, and Hispanics: A Report of the Surgeon General. Atlanta, GA: US-DHHS, US Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 1998. [Google Scholar]
- 43.Rigotti NA, Lee JE, Wechsler H. US college students’ use of tobacco products: results of a national survey. JAMA. 2000;284(6):699–705. doi: 10.1001/jama.284.6.699. [DOI] [PubMed] [Google Scholar]
- 44.Bardo MT, Donohew R, Harrington NG. Psychobiology of novelty seeking and drug seeking behavior. Behav Brain Res. 1996;77(1):23–43. doi: 10.1016/0166-4328(95)00203-0. [DOI] [PubMed] [Google Scholar]
- 45.Martin CA, Kelly TH, Rayens MK, et al. Sensation seeking, puberty, and nicotine, alcohol, and marijuana use in adolescence. J Am Acad Child Adolesc Psychiatry. 2002;41(12):1495–1502. doi: 10.1097/00004583-200212000-00022. [DOI] [PubMed] [Google Scholar]
- 46.Carton S, Jouvent R, Widloecher D. Sensation seeking, nicotine dependence, and smoking motivation in female and male smokers. Addict Behav. 1994;19(3):219–227. doi: 10.1016/0306-4603(94)90026-4. [DOI] [PubMed] [Google Scholar]
- 47.Stephenson MT, Hoyle RH, Palmgreen P, Slater MD. Brief measures of sensation seeking for screening and large-scale surveys. Drug Alcohol Depend. 2003;72(3):279–286. doi: 10.1016/j.drugalcdep.2003.08.003. [DOI] [PubMed] [Google Scholar]
- 48.Obermiller C, Spangenberg ER. Development of a scale to measure consumer skepticism toward advertising. J Consum Psychol. 1998;7(2):159–186. [Google Scholar]
- 49.Obermiller C, Spangenberg E, MacLachlan DL. Ad skepticism: the consequences of disbelief. J Advert. 2005;34(3):7–17. [Google Scholar]
- 50.Berg CJ, Stratton E, Schauer GL, et al. Perceived harm, addictiveness, and social acceptability of tobacco products and marijuana among young adults: marijuana, hookah, and electronic cigarettes win. Subst Use Misuse. 2015;50(1):79–89. doi: 10.3109/10826084.2014.958857. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Pokhrel P, Fagan P, Kehl L, Herzog TA. Receptivity to e-cigarette marketing, harm perceptions, and e-cigarette use. Am J Health Behav. 2015;39(1):121–131. doi: 10.5993/AJHB.39.1.13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Trumbo CW. The effect of electronic cigarette advertising on intended use among college students. Addict Behav. 2015;46:77–81. doi: 10.1016/j.addbeh.2015.03.005. [DOI] [PubMed] [Google Scholar]
- 53.Myslín M, Zhu S-H, Conway M. Content analysis of tobacco-related Twitter posts. Online J Public Health Inform. 2013;5(1) [Google Scholar]
- 54.Primack BA, Rice KR, Shensa A, et al. US hookah tobacco smoking establishments advertised on the Internet. Am J Prev Med. 2012;42(2):150–156. doi: 10.1016/j.amepre.2011.10.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Carroll MV, Shensa A, Primack BA. A comparison of cigarette- and hookah-related videos on YouTube. Tob Control. 2013;22(5):319–323. doi: 10.1136/tobaccocontrol-2011-050253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Krauss MJ, Sowles SJ, Moreno M, et al. Hookah-related Twitter chatter: a content analysis. Prev Chronic Dis. 2015;12:E121. doi: 10.5888/pcd12.150140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Andrews JA, Tildesley E, Hops H, Li F. The influence of peers on young adult substance use. Health Psychol. 2002;21(4):349–357. doi: 10.1037//0278-6133.21.4.349. [DOI] [PubMed] [Google Scholar]
- 58.Jackson C. Initial and experimental stages of tobacco and alcohol use during late childhood: relation to peer, parent, and personal risk factors. Addict Behav. 1997;22(5):685–698. doi: 10.1016/s0306-4603(97)00005-1. [DOI] [PubMed] [Google Scholar]
- 59.Brook JS, Whiteman M, Czeisler LJ, et al. Cigarette smoking in young adults: childhood and adolescent personality, familial, and peer antecedents. J Genet Psychol. 1997;158(2):172–188. doi: 10.1080/00221329709596660. [DOI] [PubMed] [Google Scholar]
- 60.Kear ME. Psychosocial determinants of cigarette smoking among college students. J Community Health Nurs. 2002;19(4):245–257. doi: 10.1207/S15327655JCHN1904_05. [DOI] [PubMed] [Google Scholar]
- 61.Choi WS, Harris KJ, Okuyemi K, Ahluwalia JS. Predictors of smoking initiation among college-bound high school students. Ann Behav Med. 2003;26(1):69–74. doi: 10.1207/S15324796ABM2601_09. [DOI] [PubMed] [Google Scholar]
- 62.Soneji S, Ambrose BK, Lee W, et al. Direct-to-consumer tobacco marketing and its association with tobacco use among adolescents and young adults. J Adolesc Health. 2014;55(2):209–215. doi: 10.1016/j.jadohealth.2014.01.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Purcell K, Brenner J, Rainie L. Search Engine Use 2012. Washington, DC: Pew Internet & American Life Project; 2012. [Google Scholar]
- 64.Facebook. [Accessed November 2, 2017];Ad Audiences. 2014 Available at: https://www.facebook.com/help/433385333434831.
- 65.US Food and Drug Administration. Deeming Tobacco Products To Be Subject to the Federal Food, Drug, and Cosmetic Act, as Amended by the Family Smoking Prevention and Tobacco Control Act; Restrictions on the Sale and Distribution of Tobacco Products and Required Warning Statements for Tobacco Products. [Accessed November 2, 2017];Fed Regist. 2016 81(90):28973–29106. Available at: https://www.fda.gov/downloads/AboutFD.../UCM394933.pdf. [PubMed] [Google Scholar]
- 66.Padon AA, Maloney EK, Cappella JN. Youth-targeted e-cigarette marketing in the US. Tob Regul Sci. 2017;3(1):95–101. doi: 10.18001/TRS.3.1.9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Harrell MB, Loukas A, Jackson CD, et al. Flavored tobacco product use among youth and young adults: what if flavors didn’t exist? Tob Regul Sci. 2017;3(2):168–173. doi: 10.18001/TRS.3.2.4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Lenhart A. Teens, Social Media & Technology Overview 2015. Washington, DC: Pew Research Center; 2015. [Accessed November 2, 2017]. Available at: http://www.pewinternet.org/2015/04/09/teens-social-media-technology-2015/ [Google Scholar]
- 69.Nicksic NE, Harrell MB, Pérez A, et al. Recall of e-cigarette advertisements and adolescent e-cigarette use. Tob Regul Sci. 2017;3(2):210–221. doi: 10.18001/TRS.3.2.9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Clark EM, Jones CA, Williams JR, et al. Vaporous marketing: uncovering pervasive electronic cigarette advertisements on Twitter. [Accessed November 2, 2017];PLoS One. 2016 11(7):e0157304. doi: 10.1371/journal.pone.0157304. Available at: https://doi.org/10.1371/journal.pone.0157304. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Kim A, Miano T, Chew R, et al. Classification of Twitter users who tweet about e-cigarettes. JMIR Public Health Surveill. 2017;3(3):e63. doi: 10.2196/publichealth.8060. [DOI] [PMC free article] [PubMed] [Google Scholar]