Therapeutic Claims in Cannabidiol (CBD) Marketing Messages on Twitter

Abstract

Although the U.S. FDA has only approved exactly one cannabidiol (CBD) drug product (specifically to treat seizures), CBD products are proliferating rapidly through different modes of usage including food products, cosmetics, vaping pods, and supplements (typically, oils). Despite the FDA clearly warning consumers about unproven health claims made by manufacturers selling CBD products over the counter, the CBD market share was nearly 3 billion USD in 2020 and is expected to top 55 billion USD in 2028. In this context, it is important to assess the presence of health claims being made on social media, especially claims that are part of marketing messages. To this end, we collected over two million English tweets discussing CBD themes. We created a hand-labeled dataset and built machine learned classifiers to identify marketing tweets from regular tweets that may be generated by consumers. The best classifier achieved 85% precision, 83% recall, and 84% F-score. Our analyses showed that pain, anxiety disorders, sleep disorders, and stress are the four main therapeutic claims made constituting 31.67%, 27.11%, 13.77%, and 10.37% of all medical claims made on Twitter, respectively. Also, more than 93% of advertised CBD products are edibles or oil/tinctures. Our effort is the first to demonstrate the feasibility of surveillance of marketing claims for CBD products. We believe this could pave way for more explorations into this indispensable task in the current landscape of social media driven health (mis)information and communication.

I. Introduction

Cannabidiol (CBD) is an active ingredient extracted from the cannabis sativa plant and is the second most abundant such ingredient besides tetrahydrocannabinol (THC). Although it is derived directly from the hemp plant, it does not cause euphoria associated with typical psychoactive substances and is not known to be addictive [1]. On the other hand, we don’t have substantial/conclusive evidence for many of its claimed therapeutic effects except for epilepsy [2]. The U.S. Food and Drug Administration (FDA) maintains a Web resource to inform consumers of latest developments with regards to CBD products: https://www.fda.gov/consumers/consumer-updates/what-you-need-know-and-what-were-working-find-out-about-products-containing-cannabis-or-cannabis. As per current FDA guidelines in this resource, Epidiolex is the only FDA approved CBD drug product and can be prescribed for severe forms of epilepsy. As of now, it is illegal to market CBD as a dietary supplement and any therapeutic claims are just that, given there is no substantial evidence warranting them. On the other hand, safety data regarding use of CBD products is limited and there are import risks especially liver injury, reproductive toxicity, and adverse interactions with other drugs being used by consumers. As such, it is critical to consider these FDA guidelines and consult physicians before regularly using CBD products.

Despite FDA’s circumspection, it is important to note that CBD is being given serious consideration by researchers as a potential therapy for a few conditions. A simple search on ClinicalTrials.gov surfaces nearly 300 studies looking into applications of CBD including opioid use disorder and anxiety disorder. Ongoing clinical studies also seem to indicate CBD’s potential for psychiatric conditions and chronic pain [3] but without conclusive evidence. While the scientific community pursues its agenda to rigorously evaluate therapeutic potential of CBD, after the legalization of the state-based medical cannabis and adult recreational use, the price of cannabis-based products has decreased, and their availability has increased [4]. These days, people can buy cannabis-based products in common retail markets like CVS, Kroger, or even local stores. As a result, CBD’s safety is simply assumed given it is widely available and lacks the euphoric, psychoactive characteristics of THC [5]. Although the causal relation between cannabis use and its side-effects is not clear now, frequent use of cannabis-based products is associated with some short- and long-term effects like respiratory and cardiovascular disorders, cognitive alterations, psychosis, schizophrenia, and mood disorders [6]. However, Stith et al.’s [7] study shows that CBD, in particular, did not necessarily improve any health issues but neither caused side effects based on self-reporting. But FDA’s guidelines indicate that the scientific community still does not know what happens if CBD is used daily for prolonged periods of time, in terms of its safety profile.

While the scientific community pursues the evaluation of CBD’s potential rigorously, sales of CBD products have skyrocketed to nearly three billion USD in 2020 with the market share expected to top a staggering 55 billion USD by 2028 (https://www.fortunebusinessinsights.com/cannabidiol-cbd-market-103215). This level of proliferation of CBD imbued products (oils, cosmetics, beverages, vaping pods) needs careful monitoring in terms of consumer experiences. It is also imperative to keep track of potential health-related claims being made by manufacturers of CBD products.

One medium where (mis)information can spread very rapidly is online social networks and other microblogging platforms such as Twitter. Product marketers can advertise to large audiences; consumers can also quickly share experiences about products. In recent years, these platforms have attracted the attention of researchers who study spread of information online. Particularly for some cases in public health surveillance, this type of data stream is preferred over other methods such as electronic health records, disease registries and statistics [8]. Twitter is a highly popular microblogging platform which allows people to post short messages (called tweets) within 280 characters in length [9]. There have been many studies that used Twitter’s platform to collect cannabis-related data. In [10], authors investigated the sentiment and themes of marijuana-related influential Twitter users to describe the demographics of such users. Nguyen and et al. collected marijuana-related tweets to examine users’ attitudes and compare the number of tweets posted during weekends and weekdays [9]. They evaluated Twitter users’ perceptions regarding edibles and assessed edible-related tweeting activities. One of their findings shows that Twitter data mining is a useful tool to monitor emerging drug use practices and trends [11]. Allem et al. worked on data extracted from Twitter to analyze common topics of conversation about e-liquids to inform surveillance and regulatory efforts [12]. In [13], authors examined properties of marijuana concentrate users, explained patterns and their reasons of use, and recognized factors assicuated with daily use of concentrates among U.S.-based cannabis Twitter users.

Specific to CBD and social data, Tran and Kavuluru [14] recently studied perceived or expected therapeutic uses for CBD products discussed online. They collected messages posted in the CBD Subreddit between January 1 and April 30, 2019 as well as comments submitted to the FDA regarding regulation of cannabis-derived products. They found that CBD is mostly discussed as a remedy for anxiety disorders and pain among users. Also, their results show the most popular CBD product group among consumers is oil and tinctures [14]. Leas et al. [15] showed that Google searches for CBD drastically increased since 2014 across America when they examined trends since 2004–2019.

Although already discussed studies exist at the intersection of social media and cannabis-derived products, we are not aware of any prior studies that examine marketing claims made on social media specifically with regards to CBD products. Potentially due to the focus on the COVID-19 pandemic, research into this theme has been a bit stagnant. But tracking marketing claims on social media is crucial for FDA because it explicitly warns manufacturers from making any spurious claims of that nature [16]. To demonstrate the feasibility of an automated approach to track health benefit claims of CBD products in marketing messages, in this effort, we make the following contributions.

1. From a dataset of 2,200,000 CBD focused tweets collected in July 2019, we (first two authors) hand-annotate a dataset of 1000 randomly chosen tweets as “marketing” or “non-marketing”. We build a high-quality machine learned classifier from this dataset to identify marketing messages.

2. We apply this classifier to the full dataset of tweets to identify all marketing tweets in it. We use straightforward language processing methods to identify therapeutic claims made in the marketing tweets and rank them in terms of popularity. We also identify and rank different modes of usage of CBD products.

We first needed to build the marketing tweet classifier since any genuine consumer-expressed claims are not of primary concern in this study. Our effort overall demonstrates the feasibility of surveillance of marketing health claims and can potentially help the FDA in tracking them automatically.

II. Methodology

Our methodology consists of four main tasks: data collection and annotation, preprocessing and feature extraction, classification, and finally tweet analysis. After collection of tweets, data annotation is requited so we can train a classifier with hand-curated labels (marketing vs non-marketing). In the following subsections, we elaborate on the specifics of each task.

A. Data collection and annotation

Using the Twitter API, we collected nearly 2,200,000 tweets in English language, authored during July 2019. The keywords used for crawling are “cbd”, “cbdoil” and “cannabidiol”. To build the marketing tweet classifier, we hand-annotated (first two authors) around 1000 tweets into two classes: marketing and non-marketing. We encountered a few challenges in the process of labeling the tweets. Tweets that were too short or contain only hashtags were ambiguous in terms of our classification problem. The Cohen’s kappa inter annotator agreement was 0.43 in the first round of annotations, which indicates “weak” agreement [14]. The annotators discussed the examples causing the disagreements and refined the annotation guidelines. To obtain high quality labels, the annotators worked on two small batches of 100 tweets each to further refine the guidelines as is standard practice. Subsequently, they annotated a new round of 1000 tweets. Cohen’s Kappa score this time around is 0.67 for the new (second) annotation set which shows moderate score of agreement [14]. This new set of annotations was used to train the marketing tweet classifiers.

B. Preprocessing and feature extraction

After collecting the data, we preprocessed tweets to ignore extremely short uninformative tweets and tweets written in any language other than English (even though we imposed a constraint of English-only tweets when using the API, transliterations from other languages and code-switching tweets naturally creep into the dataset). Duplicate tweets were removed because in this initial stage it is not important to examine separately tweets that look identical. Any other preprocessing tricks that are typically applied for text classification are used (e.g., lowercasing all tokens).

We extracted the well-known unigram and bigram features from each tweet [17]. These are individual tokens and adjacent token pair features that are commonly used in text classification. We also included two hand-crafted features: a Boolean feature that indicates the presence of a URL in the tweet and another Boolean feature capturing the presence of “cbd” in the username of the tweet’s author.

C. Classification

To evaluate the features extracted in the previous section, we have selected two well-known algorithms namely Support Vector Machines (SVM) and Logistic Regression (LR). 70 percent of annotated tweets are used for training and the rest of them for testing. The length of the n-gram feature vector along with the other hand-crafted features is 14,799. These feature vectors are fed to each classifier as input. In the Results and Discussion section, we will compare the performance of various classifiers.

D. Tweet Analysis for therapeutic claims

After training the classifier and applying the trained classifier on unseen tweets, we will have two categories of tweets, marketing and non-marketing. Here, we aim to mostly concentrate on marketing tweets so that we can more effectively analyze unsubstantiated health claims of companies and their representatives. To ensure precise analyses, we only consider a window of 72 characters of each tweet conveying a CBD-based therapy claim. This is accomplished by making sure that these three terms are within the 72-character window: CBD-related keywords, condition/disease terms (CT), and therapeutic trigger phrases. CBD-related keywords are the same as the keywords used to collect tweets: “cbd”, “cbdoil”, and “cannabidiol”. CT terms indicate conditions we want to explore such as pain, anxiety, and autism. We chose the ten conditions (more in the next section) we studied in our previous study of consumer messages [14]. This was done to compare the relative rankings of claims in consumer messages and marketing messages. Finally, therapeutic trigger phrases include terms that indicate a therapeutic potential in genera. These phrases are extracted manually from a dictionary based on our prior work [14]: “treats”, “cures”, “helps”, “reduces”, “alleviates”, “relieves”, “eliminates”, “kills”, “stops”, “eases”, “aids”, “soothes”, “inhibits”, “improves”, “destroys”, “reverses”, “suppresses”, “lowers”, “regulates”, “prevents”, “manages”, “fixes”, “better”, “benefit”, “effective”, “useful”, “decrease”, “solve”, and their all variations. In order to reduce the risk of misleading negated results, tweets with terms like “not”, “no”, “never” are excluded. Generally, three mentioned restrictions are applied on each document and then we count the frequencies of each event. For example, for the condition term “pain”, we will count such a document in which three mentioned terms happened in the defined window: “@elzbthrmn are you suffering from deep muscle aches amp joint pains i have three options try a massage using cbd therapeutic rub to reduce inflammation eliminate pain amp suppress muscle spasms cbd free education amp purchase @ https t co lp9zvrr1yx cbd cbdoil cbdheals cannabidiol https t co 6latn4juim.”

A related aspect we wanted to study is the common types of CBD-based products advertised in marketing messages. To handle this, different consumption methods of CBD-based products are divided into five groups, including oils/tinctures, vapes, edibles, pills/capsules, and topicals [14]. We determine (manually) various terms for each category to define each of them precisely. Table I shows the terms used to indicate each category. Defined terms in each group are checked for all tweets and then frequency of the terms in each group reflects the popularity of that product type in marketing tweets.

TABLE I.

Five general groups of CBD-based products

Product Group	Defined terms for matching
Oil/Tinctures	“oil”, “tincture”, “drops”, “droplets”
Vapes	“vape”, “juice”, “dab”
Edibles	“edible”, “gummy”, “gummies”, “coffee”, “candy”, “tea”, “chocolate”, “drink”, “honey”, “cookie”, “snacks”, “brownies”, “syrup”, “beverage”, “chews”, “smoothies”
Pills/Capsules	“pills”, “capsules”, “caps”, “softgels”
Topicals	“cream”, “salve”, “balm”, “topicals”, “lotion”, “patches”, “gel”, “ointment”, “topical cream”, “body butter”

III. Results and Discussion

As discussed earlier, our main aim is to find a relative ranking of popular therapeutic claims of CBD products as asserted or implied in marketing tweets. Before analyzing the marketing tweets, we need to distinguish marketing tweets from non-marketing tweets. To this end, we first needed a way to identify marketing tweets for which we build basic LR and SVM models as discussed in the previous section.

Table II lists the results of classifiers. We can see that the SVM model is slightly better than the LR model, with 85.02% accuracy, 0.85 precision, 0.83 recall, and 0.84 F-score. Hence, we use this model for the rest of our analyses. It is important to note that the precision of 85% is important for us because we want to make meaningful conclusions about claims being made in marketing tweets and hence too many false positives would be detrimental to the outcomes.

TABLE II.

Results of different classifiers with n-gram Features

Classifier	Accuracy (%)	Precision	Recall	F-score
SVM	85.02	0.8526	0.8331	0.8401
LR	84.58	0.8489	0.8276	0.8350

Table III shows that we have a total of 1,205,224 tweets out of the full dataset of 2,200,000 tweets after preprocessing was completed as per section II (B). Of these preprocessed tweets, 50.37% (607,122 tweets) were classified as marketing tweets and remaining 49.63% (598,102 tweets) as non-marketing tweets when we apply our classifier to all such tweets. Although a nontrivial amount of marketing chatter is expected, it is surprising to see that more than half of the tweets are marketing related. Even if we only consider 85% of these tweets (owing to 0.85 precision), it is still over 42% of the dataset containing marketing of CBD products.

TABLE III.

Statistics of Marketing and non-marketing tweets after preprocessing and classification

Total Tweets	After Preprocessing	Marketing Tweets	Non-Marketing Tweet Num
2200000	1,205,224	607,122 (50.37%)	598,102 (49.63%)

Table IV consists of top ten different main conditions listed in the first column covered by various variant terms in the second column. The third column indicates the number of tweets where a CBD term, a therapeutic phrase, and a condition term (2^nd column) are mentioned in a window of 72 characters indicating an overall therapeutic claim. The final column of the table shows various example tweets from the dataset that give the reader an idea of the claims being made. As can be observed, the example claims are mostly direct with little nuance or ambiguity, which is very surprising. Also, marketing tweets that make these claims appear to group many conditions within the same message.

TABLE IV.

The frequency of therapeutic claims made with different condition terms in marketing tweets as per the windowing constraint of having a cbd term, a condition, and a therapeutic phrase in 72 consecutive characters in the tweet

Condition Terms	Variant Terms	Therapeutic claim count	Example tweets that contain a therapeutic indication for a CT (column 1)
Anxiety Disorders	anxiety, anxious, panic attacks, GAD, panic disorder, anxiousness	4858	Ex1: CBD products can help you get a good night’s sleep. #CBD helps reduce anxiety and stress, which impact sleep. Learn how CBD can help you dream the night away. https://t.co/ITEDO3raIc #sleepdisorders https://t.co/7ZRpYRJ4NJ Ex 2: Full spectrum CBD, reduce anxiety and increase happiness! #cannapresso #cannapressocbd #cannapressocbdoil #CBD #cbdoil #cbdfullspectrum #fullspectrumcbd #cbdwater #cbdlife #cbdoils #cbdspray #cbdoillvape https://t.co/krDgmZYRS0
Pain	pain, painful, ache	5675	Ex1: Enjoy a pain-free and anxiety-free summer with our CBD products! We are here to help! Please visit our online store: https://t.co/LpFkMNrzZ8 For more information about our products please visit https://t.co/oaYflwhQ7S Ex2: Ever wonder what types of pain our CBD Pail Gel can potentially help? https://t.co/Zu5AL9QbOV : @purewaterhealth #NaturesScript https://t.co/ouUFUTWgAk
Depressive Disorders	depression, depressed, miserable, depressing, misery	515	Ex1: Our #CBD Oil Tinctures help ease pain, reduces stress, anxiety and depression, improves heart health and encourages deeper, more peaceful sleep. Learn more (and shop!) at https://t.co/1bgmyM3z67 https://t.co/c5686nQ48e
Inflammation	inflammation, inflammatory, reaction	1146	Ex 1: DYK an increasing number of studies support the benefits of CBD for ailments such as pain, inflammation, sleep & nausea? We believe Canadians should have better access to CBD to #GetWellNotHigh. Visit https://t.co/oGfeLB2zmu to tell your MP you want #CBD as an #NHP. https://t.co/PdZYOTNmpI Ex2: Products like our CBD balm may help target and manage pain and inflammation in specific areas of the body. https://t.co/8RXgXxR0XI
Stress	stress, stressful, stressed, stressing, stresses	1859	Ex1: @Nadtastik Yes it works. Here are the Benefits of CBD Tea: Reduces Stress, Anxiety, And Depression Boosts Mood Promotes Health And Wellness Lowers The Levels Of Cholesterol Supports Excellent Immune Functioning Revives The Body Read more from - https://t.co/TVJXZ7Je4N Ex 2: Our #CBD Oil Tinctures help ease pain, reduces stress, anxiety and depression, improves heart health and encourages deeper, more peaceful sleep. Learn more (and shop!) at https://t.co/1bgmyM3z67 https://t.co/c5686nQ48e
Headache	“headaches”, “migraines”	163	Ex 1: The perfect cure for headaches new product launch soon #unmade #cbd #headaches https://t.co/2mN9b3Ex5t
Sleep Disorders	“insomnia”, “sleep”, “sleeping”, “sleepless”, “sleeplessness”	2468	Ex 1: Our Sleep Support Spray is combined with a special blend of #CBD and other important nutrients to help promote a long & restful sleep. https://t.co/EL4mSKz9Gg https://t.co/bPl4ItwAI8 Ex 2; Our Feel Peace line of CBD products combines the benefits of broad-spectrum CBD with the calming effects of chamomile & melatonin to help you rest and rejuvenate. #cbd #cbdoil #sleepaid #restlessness #insomnia https://t.co/NFgXpdmAeU
Seizure Disorders	“seizures”, “epilepsy”, “epileptic”	400	Ex 1: CBD oil for dogs! Yep! CBD oil can very likely help treat seizures, stress, anxiety, arthritis, back pain, symptoms of cancer, and gastrointestinal issues, among other health conditions in dogs. #cbdoil #cbdfordogs #cannabidiol #endocannabinoidsystem #anxietyrelief #stressrelief https://t.co/A5a5dHFKLQ
Cancer	“cancerous”, “cancer”	700	Ex 1: How Rick Simpsons Oil Works to Kill Cancer Cells https://t.co/oGEat0cF6v #Cancer #RSO #CBDOIL #arrorandkimwarer #TuesdayThoughts #TuesdayMotivation #RotichError #InteriorKazini Ex 2: Order via - info@ricksimpsonsoil.com: CANNABIS Oil FIGHTS CANCER AND HELPS MANAGE ASPECT… https://t.co/It1V1yPmq1 #Cancer #RSO #CBDOIL #RIPMugabe #UoNgraduation #NotAnInchLess BREAKING NEWS
Nausea	“nausea”, “nauseous”, “nauseated”, “nauseating”	130	Ex 1: Our gummies are packed with flavor and infused with 20mg of our purest CBD hemp oil. CBD has been shown to reduce nausea, suppress muscle spasms, reduce seizures, decrease stress and anxiety, promote relaxation and overall health. $39.99 https://t.co/du1uyVzKIn https://t.co/Diea3O5XLx

Furthermore, although the only FDA approved CBD drug product is for seizures, as per our Twitter data, it ranks 8 out of 10 in terms of frequency of associated claims. The example message shown for seizures also appears to claim benefits for dogs. Overall, pain, anxiety disorders, sleep disorders, and stress constitute 31.67%, 27.11%, 13.77%, and 10.37% of the top ten conditions reported, respectively; pain is the most frequent claim made overall. In our prior study with consumer Reddit posts, anxiety was the top claim followed by pain, depressive disorders, and inflammation [14].

Similarly, we also counted the number of matches for different types of products (from Table I) in marketing tweets. The two common types of CBD items are edibles (75%) and oils/tinctures (18%) as shown in Figure 1. It is surprising to see that edibles dominate substantially. However, the choice of imbuing food products with CBD may make the products more appealing to many people because there is additional satisfaction of consuming food.

Fig 1:

Distribution of CBD-based product types

In Table V, we show the numbers of therapeutic messages and associated unique tweeters in both marketing and non-marketing subsets. It is a bit surprising to see there are many more health claims in the non-marketing subset than the marketing tweets even though these two subsets are nearly equal in size. This indicates that even regular users are talking about therapeutic effects of CBD products. However, when we look the unique tweeters, there are far fewer in the marketing tweet claims. The table also shows that on average the tweeters who created the claims in the marketing subset generate nearly two such tweets per tweeter. However, this ratio is 1.27 in the non-marketing group. This means that a select number of people may be responsible for many of the marketing health claims.

TABLE V-.

The ratio of therapeutic tweets to unique user IDs for both marketing and non-marketing tweets

	Marketing tweets	Non-marketing tweets
Unique Twitter user IDs	3538	8931
Unique therapeutic tweets	6854	11358
The ratio of unique therapeutic tweets to unique tweeters	1.937	1.27

IV. Conclusion

In a first of its kind study, we analyzed the presence of therapeutic claims for CBD in marketing messages on Twitter. To do this we first created a hand-labeled dataset and built a classifier to identify marketing messages from CBD themed tweets. The classifier was subsequently applied to the full corpus of over a million CBD tweets. This surfaced our first main finding that over 50% of CBD tweets appear to be marketing related chatter. We then identified therapeutic messages in the marketing tweets using a pattern-based approach that spots the presence of a condition, a therapeutic effect indicating term, and a CBD term in a small window of text. We chose a set of ten popular conditions that are cited by consumers for their CBD usage in our prior work [14]. Pain and anxiety are the most popular conditions discussed by marketing messages. Edibles are by far the most popular product type being advertised, followed by oils.

Our work is not without limitations. We assume that the presence of a CBD term, a condition, and a therapeutic trigger phrase in a small window of text implies that a health benefit claim is being made. This may not always be true. However, our prior work indicates this is true 95% of the time [14] and hence our results are reasonably meaningful. Also, the precision of the marketing tweet classifier is only 85% and ought to be improved to get better estimates reported in claims made. Moreover, our study analyzes only Twitter data during a particular duration in 2019. Other social media (Instagram, Facebook) and data collected across longer durations may change some of our estimates. Overall, however, we believe the results are strong enough to demonstrate feasibility of surveillance of CBD marketing claims. The example claims shown in the results section are clearly blatant sales pitches that disregard FDA’s general guidance on this topic. Considering this, we hope more researchers will pursue this line of work in future.