Identifying E-cigarette Content on TikTok: Using a BERTopic Modeling Approach

Abstract

Introduction

The use of hashtags is a common way to promote e-cigarette content on social media. Analysis of hashtags may provide insight into e-cigarette promotion on social media. However, the examination of text data is complicated by the voluminous amount of social media data. This study used machine learning approaches (ie, Bidirectional Encoder Representations from Transformers [BERT] topic modeling) to identify e-cigarette content on TikTok.

Aims and Methods

We used 13 unique hashtags related to e-cigarettes (eg, #vape) for data collection. The final analytic sample included 12 573 TikTok posts. To identify the best fitting number of topic clusters, we used both quantitative (ie, coherence test) and qualitative approaches (ie, researchers checked the relevance of text from each topic). We, then, grouped and characterized clustered text for each theme.

Results

We evaluated that N = 18 was the ideal number of topic clusters. The 9 overarching themes were identified: Social media and TikTok-related features (N = 4; “duet,” “viral”), Vape shops and brands (N = 3; “store”), Vape tricks (N = 3; “ripsaw”), Modified use of e-cigarettes (N = 1; “coil,” “wire”), Vaping and girls (N = 1; “girl”), Vape flavors (N = 1; “flavors”), Vape and cigarettes (N = 1; “smoke”), Vape identities and communities (N = 1; “community”), and Non-English language (N = 3; Romanian and Spanish).

Conclusions

This study used a machine learning method, BERTopic modeling, to successfully identify relevant themes on TikTok. This method can inform future social media research examining other tobacco products, and tobacco regulatory policies such as monitoring of e-cigarette marketing on social media.

Implications

This study can inform future social media research examining other tobacco products, and tobacco regulatory policies such as monitoring of e-cigarette marketing on social media.

Introduction

E-cigarette use increases health risks for young people through exposure to nicotine and other toxicants.¹ In 2022, 11.3% of U.S. youth reported e-cigarette use in the past 30 days.² High rates of e-cigarette use are driven in part by exposure to e-cigarette content on social media.³ E-cigarette content on social media is abundant, often featuring engaging and appealing themes for young people, such as portraying e-cigarettes as “healthy” and “glamourous.”⁴ E-cigarette content on social media is quickly evolving so it is important to use innovative tools to assess pro-e-cigarette content on social media.

The use of hashtags is one promotional strategy used to promote e-cigarettes on social media.⁴ Hashtags (annotated with #) are used to index keywords or topics on some social media (eg, Twitter [X], Instagram, TikTok), allowing users to easily identify and follow topics they are interested in. Common e-cigarette-related hashtags include communities related to e-cigarette use such as #vapefam, and #vapenation.⁴ Other examples of e-cigarette-related hashtags include features of e-cigarette products (eg, flavors, names of vape brands, and vape shops) and e-cigarette-related content (eg, vape tricks). E-cigarette-related hashtags can attract users and be used to promote e-cigarettes on social media. Examining the text data (eg, use of hashtags) on social media is important to better understand the types of e-cigarette-related content currently available on social media.

Examination of e-cigarette-related text data (eg, hashtags) is complicated by the voluminous amount of text data on social media. In response to these complexities, machine learning has been increasingly used to identify and examine e-cigarette-related content on social media.⁴ In particular, topic modeling approaches have been used to identify e-cigarette-related text on social media, including hashtags.^5,6 Specifically, Bidirectional Encoder Representations from Transformers (BERT) topic modeling can identify and cluster e-cigarette-related text-based content on social media with high levels of accuracy.^5,7 BERTopic modeling is an unsupervised machine learning technique based on Google’s neural network-based technique that uses natural language processing pretraining to transform text-based social media data into interpretable topics,⁵ enabling identification of the most common e-cigarette topical themes on social media. Traditionally, topic modeling such as latent Dirichlet allocation has been based on a “bag of words” approach, wherein statistically cooccurring words are grouped together.⁸ However, the context of words can easily be lost in such techniques. BERTopic uses Google’s pretrained BERT model to generate document-level topic information, organized by topic clusters. Unlike methods such as latent Dirichlet allocation, BERTopic retains context, blending classical statistical machine learning with next-generation neural-based methods. Moreover, BERTopic has integrated visualization libraries to enable greater human interpretability of machine-generated clusters.⁵

A previous study by Baker et al.⁵ used BERTopic modeling to identify clusters of e-cigarette-related content on Twitter (X) including promotional e-cigarette content (ie, advertisements) and first-hand experiences of e-cigarette use shared by Twitter (X) users, indicating the utility of BERTopic modeling approaches to successfully and accurately identify and cluster e-cigarette-related content on Twitter (X). This finding indicates that BERTopic modeling can be utilized to monitor social media platforms for e-cigarette use trends by enabling efficient and timely distillation of social media text data. However, it has not been used to examine e-cigarette content on TikTok, the most frequently used social media platform by youth and young adults in 2022.⁹ Examining e-cigarettes on TikTok is important because TikTok is the most frequently used social media platform (60%) by youth (ages 13–17),¹⁰ and the use of TikTok has drastically increased among adults from 2021 to 2023.¹¹ Previous research showed that youth who used TikTok more frequently are more likely to initiate e-cigarette use than youth who used TikTok less frequently.¹² Despite the popularity of TikTok among young people and its role in e-cigarette initiation, tobacco content on TikTok is relatively understudied than other social media platforms such as Instagram, Twitter (X), and Reddit. Even though TikTok is primarily a video-based platform, TikTok users commonly communicate and interact with other users via text such as hashtags, including the use of strategic hashtags to increase content exposure and interest among other TikTok users. Therefore, this study aimed to use BERTopic modeling, a machine learning approach to identify and examine e-cigarette content on TikTok.

Methods

Data Collection and Processing

We used 13 unique hashtags related to e-cigarettes (eg, #vape, #vapelife, and #vapenation) for data collection conducted in February 2022. The hashtags were identified as the most used hashtags related to vaping on TikTok.¹³ We used a Python script¹⁴ to scrape videos and textual data (eg, hashtags) from TikTok API. We extracted the top 1000 posts from each hashtag, unless the hashtag had fewer than 1000 results. This represents a robust sample of TikTok posts in each hashtag. The final analytic sample included 12 573 TikTok posts. We first completed a corpus tuning step, which included the removal of: stopwords (eg, “the,” “a,” “an,” “what,” and “so”), and TikTok-related hashtags like “#fyp” (is short for “for you page” and is primarily used to increase visibility¹⁵).

To identify the best fitting number of topic clusters, we used both quantitative (ie, coherence test) and qualitative approaches (ie, researchers checked the relevance of texts from each topic to e-cigarette-related content). We identified N = 18 topic model offers the highest coherence score as well as interpretability of topics. See Figure 1 for flow chart of the data collection process and data processing and quantitative/qualitative analyses process.

Figure 1.

Flowchart of data collection and data processing using BERTopic modeling.

Quantitative and Qualitative Analyses

To identify the best-fitting number of topic clusters, we used both quantitative and qualitative approaches. Coherence test scores represent the distance between words that are part of a topic. We computed coherence test scores of our BERTopic model using the UMass coherence model¹⁶ from the Gensim Python library.¹⁷ Higher coherence test scores indicate higher consistency across words within the topic clusters. We evaluated which topic outputs had high coherence scores.

Then, 2 PhD-level researchers checked the relevance of texts from each topic to e-cigarette-related content while checking for consistency in e-cigarette content within each topic. Specifically, these checks included viewing the TikTok videos and examining the texts in posts associated with identified texts, and searching for identified terms online to confirm that they are e-cigarette-related (eg, e-cigarette brand or device names) to group and characterize each theme. One coder did the categorization, and the other coder confirmed the groups. When the first coder was unsure about grouping, topics were grouped into overarching themes collaboratively via discussion between the two coders. When grouping topics, the researchers focused on maximizing thematic consistency within each theme and minimizing overlaps in topics across themes. The researchers then drafted a description of each theme that reflected the shared content across topics within that cluster. We finally decided on the best fitting number of topic clusters by evaluating across quantitative (ie, coherence test scores) and qualitative (ie, consistency of words within each topic) assessments.

Results

Coherence Test

The coherence test showed the highest score (Figure 2), indicating higher consistency in words/texts within topics, in N = 7, followed by N = 10, N = 20, N = 23, and N = 18. After iterative processes of quantitative and qualitative analyses based on the coherence test (ie, we qualitatively reviewed the content of the videos that appeared in topics N = 7, N = 10, N = 20, N = 23, N = 18 topics), we found that N = 18 was the ideal number of topic clusters qualitatively. We grouped similar topics into 9 overarching themes (Table 1). We also included Table S1 showing the results of other BERTopic modeling versions, N = 7, N = 10, N = 25, and N = 23 (Table S1).

Figure 2.

Results of coherence test.

Table 1.

Results of BERTopic Modeling

Themes	Topics (N = 18)	Example hashtags
1. Social media and TikTok-related features	N = 4	“duet,” “viral,” “trending,” “comedy,” “skit”
2. Vape shops and brands	N = 3	“store,” “drag” (product name from a UK vape brand)
3. Vape tricks	N = 3	“ripsaw,” “dragon,” “bane,” “lasso”
4. Modified use of e-cigarettes	N = 1	“coil,” “wire,” “build”
5. Vaping and girls	N = 1	“girl,” “girls”
6. Vape flavors	N = 1	“flavors”
7. Vape and cigarettes	N = 1	“smoke,” “fume”
8. Vape identities and communities	N = 1	“community,” “nation,” “family”
9. Non-English topics	N = 3	Hashtags in other languages Romanian and Spanish

Qualitative Analyses of E-cigarette-Related Topics

Four topics fit into the “social media and TikTok-specific features” (eg, “duet,” “viral,” “trending,” and “comedy”) theme. These texts are not e-cigarette specific but may be used to increase engagement with e-cigarette-related posts. For instance, hashtags such as “comedy” and “skit” included a short video showing a popular comedy TV show featuring e-cigarettes. Hashtags “viral” and “trending” are widely used across TikTok content to increase attention to specific posts, potentially increasing exposure of e-cigarette content to TikTok users. The “duet” is a type of post that shows two TikTok posts in parallel. These are frequently used for reaction videos (eg, showing some interesting video or viral topic along with the poster’s face as they react to it). For instance, one video showed a vaping product and the words “duet with us,” encouraging people to engage with their content.

Three topics fit into the “vape shops/brands” (eg, “store,” “drag” [product name from a UK vape brand called VooPoo]) theme. The “store” hashtags showed TikTok videos featuring and showing vape shops to the public and commonly used brand names in hashtags. These videos often displayed arrangements of vape products and flavors. TikTok posts even provided a physical address and contact number of the vape shops. Additionally, three topics fit into the “vape tricks” theme. The texts featured specific vape tricks such as, “ripsaw,” “dragon,” “bane,” and “lasso.”¹⁸ These vape trick videos were visually (eg, using filters and bright lights) and audibly stimulating (eg, use of background music such as hip hop, rock, or funk). Most of these videos featured young people and portrayed vape tricks as cool, stylish, and fashionable.

Other overarching themes included modified use of e-cigarettes (N = 1 topic; “coil,” “wire,” “build”), vaping and girls (N = 1 topic; “girl,” “girls”), vape flavors (N = 1 topic; “flavors”), vape and smoke (N = 1 topic; “smoke,” “fume”), and vape identities and communities (N = 1 topic; “community,” “nation,” “family”). The “modified use of e-cigarettes” theme related to using texts related to device components such as “coil,” “wire,” and “build.” These videos showed e-cigarette products and featured how to modify vaping devices. The “vaping and girls” theme featured young women who vape, and frequently used hashtags such as “girl.” The “vape flavors” theme commonly showed arrangements of flavored vaping products, introduced new vape flavors, and was frequently accompanied by the promotion of vape shops and brands. This theme commonly used text “flavor.” The “vape and cigarettes” theme featured alternative uses for e-cigarettes and cigarette smoking and frequently promoted e-cigarettes as a cigarette smoking cessation aid. This theme commonly used text such as “smoke” and “fume.” The “vape identities and communities” theme included e-cigarette promotional content and used texts to insinuate a sense of community among individuals who vape and to identify individuals already within the vaping community such as “community,” “nation,” and “family.” We also identified three topics that were not related to vaping and not in English (eg, Romanian, Spanish). For example, one uploader posted several videos showing the Pacific Ocean from his boat, using the text “vapor,” which means “ship” in Romanian.

Discussion

This study used a state-of-the-art neural machine learning method, BERTopic modeling, to successfully identify relevant themes on TikTok. Consistent with other social media platforms,^4,19–21 we found prominent e-cigarette-related topics on TikTok by examining the use of e-cigarette-related texts (eg, hashtags). Overarching themes included texts reflecting social media and TikTok-related features, vape shops and brands, vape tricks, e-cigarette device modification, targeted populations (ie, girls), vape flavors, featuring and promoting e-cigarettes as a smoking cessation aid, and highlighting “sense of belonging” among e-cigarette users. These findings are particularly concerning given that 67% of U.S. teens (ages 13–17) use TikTok⁹ and may be exposed to this content.

Social media platforms have made efforts to monitor and regulate e-cigarette-related content on their platforms, including age restrictions, and a ban on paid promotions.²² However, such restrictions and policies may not be sufficient without appropriate regulations.^19,23 For instance, TikTok’s policy states that “We do not allow showing or promoting recreational drug use, or the trade of alcohol, tobacco products, and drugs.” and “We do not allow showing or promoting young people possessing or consuming alcohol, tobacco products, and drugs.”²⁴ Nonetheless, this study found frequent presence and promotion of e-cigarettes and marketing content on TikTok. Continuous monitoring of such marketing content and strong enforcement of social media policies restricting promotional content are needed.

The identified themes are particularly concerning given that e-cigarette-related content, particularly marketing content, frequently uses youth-appealing themes such as vape tricks, e-cigarette modification, and engaging hashtags such as “viral and “comedy.” For instance, example videos with “comedy” hashtag were from a popular comedy show or TV drama, which may be used to attract young viewers to content.²⁵ Furthermore, duet videos on TikTok could be used as a social media engagement tactic by the e-cigarette industry. Many of the identified posts using the “duet” hashtag and feature were uploaded by vape shop retailers, often featuring e-cigarette devices and brands and encouraging TikTok users to engage (ie, create a duet) with the content. This is particularly concerning since engaging with tobacco content on social media is associated with tobacco use behaviors.^26,27 Hashtags such as “duet” and “comedy” are therefore example hashtags worth additional surveillance when identifying and regulating novel e-cigarette marketing strategies on popular platforms such as TikTok.

We also identified topics related to “vaping and girls,” including depictions of young women who vape. This is particularly concerning due to the high frequency of girls who use TikTok (ie, 73% of teen girls aged 13–17 reported using TikTok in 2022).⁹ E-cigarette content on social media uses specific texts to promote and normalize e-cigarette use among girls. For instance, “juulgirls” was identified in previous studies as one of the most frequent e-cigarette-related hashtags used on Instagram, normalizing vaping among girls.²⁸ Furthermore, e-cigarette-content commonly uses appealing themes that may appeal to girls such as portraying e-cigarette use as glamourous and depicting female celebrities and social media influencers in fashionable styles that are geared toward girls.⁴ Future research should examine how particular types of e-cigarette content may appeal to girls on TikTok and whether the use of these texts actually results in more exposure and engagement with e-cigarette content among girls.

Furthermore, we identified topics related to “vape flavors,” including promotions of flavored vaping products. Vape flavors have been commonly featured on other social media platforms such as Instagram, Twitter, and YouTube, particularly with e-cigarette marketing content.⁴ For example, one large-scale machine learning study found that 15.4% of e-cigarette content on Instagram featured flavored e-liquids and 58% of them were posted by e-cigarette industries.²⁹ Another large-scale machine learning study found that 14.7% of e-cigarette content on YouTube featured flavored e-liquid and 57.5% of flavored e-liquid videos included purchasing links for these products.¹⁹ This is concerning since flavors are a primary reason reported by youth and young adults for using e-cigarettes.^30,31 Promotion of flavors on TikTok may therefore tap into the appeal of flavors among youth, contributing to youth e-cigarette use behaviors. Future studies should examine types of e-cigarette flavor-related content and examine how such content might appeal to young people and influence their decisions to use e-cigarettes.

We also identified topics related to “vape and cigarettes,” commonly featuring and promoting e-cigarettes as a cigarette smoking cessation aid. Featuring e-cigarettes as a potential smoking cessation aid has been common on other social media platforms.⁴ The effectiveness of e-cigarettes as a smoking cessation aid is still unclear, and the long-term health effect of e-cigarette use is still unknown.³² Given that, promoting e-cigarettes as a smoking cessation aid may be misinformation as this is not yet confirmed by evidence. E-cigarette use may also be associated with other negative consequences such as the increased risk of other substance use (eg, vaping cannabis) and exposure to toxicants and chemicals not meant for inhalation.³³ Thus, continuous monitoring of content promoting e-cigarettes as a cessation aid is important for limiting misinformation. Furthermore, developing and implementing counter-messaging on social media with information about e-cigarettes is warranted.

To the best of our knowledge, this is the first study to use topic modeling approaches to identify e-cigarette-related content based on texts in TikTok. Nonetheless, this study has limitations. First, it only captured TikTok videos during February 2022. Given the rapidly changing content of social media, monitoring of e-cigarette-related TikTok content is warranted. Second, three non-English topics were identified even though we restricted the study to English-language videos. This is due to the inclusion of hashtags in other languages (eg, Spanish, French, Romanian) that contain words also present in the English language (eg, vapor) but that have different meanings. Future research should consider this, and qualitative cross-validation is recommended. Third, we only used text data (ie, hashtags), which does not represent audio or visual components of vape-related content on TikTok. However, understanding text information such as hashtags on TikTok is still informative. Using hashtags is a key promotional strategy to promote e-cigarettes on TikTok. As hashtag allows TikTok users to easily identify and follow topics they are interested in, hashtags in e-cigarette-related TikTok content can attract and be used to promote e-cigarettes to TikTok users. However, given that TikTok has audio and visual features, future studies should leverage these data to enhance surveillance of e-cigarette content on TikTok. Fourth, we did not calculate the relevance score or the proportion of irrelevant content pre- or post-BERTopic modeling following other topic modeling studies of e-cigarettes. Rather, we used topic modeling to determine what topics were relevant to e-cigarettes consistent with a previous topic modeling study.⁵ However, the significance of our study is the use of unsupervised machine learning to provide information on what text data (ie, hashtags) were used in the vape-related content on TikTok. Fifth, we did not systematically examine cooccurring terms and hashtags in the current study. BERTopic modeling is used to identity topics and themes across hashtags but does not examine cooccurrence. Future work is warranted to examine the cooccurring hashtags to identify how hashtags are presented together to promote e-cigarette content on TikTok.

Despite the limitations, the innovative topic modeling approach used in the current study can be helpful in providing a quicker, in-the-moment understanding of what e-cigarette-related texts are being discussed on social media. Through the use of machine learning techniques, such as BERTopic modeling, we can distill large amounts of information to detect emerging trends and themes. We provide a few methodological recommendations to guide the use of this approach. During the computational process, the researchers are recommended to review the collected data and remove hashtags unrelated to e-cigarettes but which are overly used to raise the visibility of the content on TikTok such as “#fyp” (is short for “for you page” and is primarily used to increase visibility¹⁵) to reduce the noise of the dataset. During the qualitative analysis process to identify the themes of clustered hashtags, researchers are recommended to view example posts from each topic to confirm content and its relevance to tobacco control. Overall, future social media research and tobacco regulatory efforts in regular and timely surveillance can focus on checking to see if new themes of tobacco marketing and use trends emerge and how the trends are changing such as whether the content is targeting new audiences, promoting new brands or new device types. This method can improve surveillance to inform the regulation of tobacco content on social media to create a healthier online environment for young people.

Supplementary material

Supplementary material is available at Nicotine and Tobacco Research online.

Funding

Research reported in this publication was supported by National Institute of Drug Abuse (NIDA) and FDA Center for Tobacco Products (CTP): R01DA049878 (PI: Kong, G). Rachel Ouellette’s effort was supported by grant number 5T32DA019426-18 (PI: Tebes) from the National Institute on Drug Abuse (NIDA). Grace Kong’s and Juhan Lee’s effort was also supported by grant number U54DA036151 from the NIDA and FDA CTP. The content is solely the responsibility of the authors and does not necessarily present the official views of the NIH or the Food and Drug Administration.

Declaration of Interest

None.

Author Contributions

Juhan Lee (Investigation [equal], Writing—original draft [equal], Writing—review & editing [equal]), Rachel Ouellette (Investigation [equal], Writing—review & editing [equal]), Dhiraj Murthy (Data curation [equal], Formal analysis [equal], Methodology [equal], Supervision [equal], Writing—review & editing [equal]), Ben Pretzer (Data curation [equal], Formal analysis [equal], Methodology [equal]), Tanvi Anand (Data curation [equal], Formal analysis [equal], Methodology [equal]), and Grace Kong (Funding acquisition [equal], Investigation [equal], Supervision [equal], Writing—review & editing [equal])

Data Availability

Data are available upon reasonable request. Data used from this study are publicly available data from TikTok. However, we can provide data upon reasonable request.