A mixed-methods study to inform the clarity and accuracy of cannabis-use and cannabis-tobacco co-use survey measures

Abstract

Background:

Legalization of medical and recreational cannabis in US states has been accompanied by increases in availability, acceptability, and diversity in methods of cannabis use, as well as an increase in devices and methods for cannabis-tobacco co-use. Updated and specific survey measures of cannabis and cannabis-tobacco co-use are needed.

Methods:

We employed a mixed-methods approach to identify sources of specification and measurement error in cannabis and cannabis-tobacco co-use measures. We surveyed and interviewed 36 young adult (age 18–29) cannabis and tobacco co-users in California (2017–2018), triangulated with document analysis of online cannabis websites and forums. We investigated how survey reports of cannabis use and cannabis-tobacco co-use compare to narrative descriptions provided during in-depth interviews. We identify key strategies for researchers collecting self-reported survey data to enhance accuracy and comprehensiveness of measures.

Findings:

Potential sources of survey error included: broad variation in cannabinoid content and concentration, inconsistent interpretation of questions between participants and researchers (e.g. blunts were not considered co-use), and substantial variation in dosage within and between products. No evidence of survey recall bias or response editing was detected.

Conclusion:

To enhance survey accuracy, we recommend surveys specify which cannabis delivery methods and forms are included and excluded in each measure, differentiate between cannabis products, and explicitly include or exclude CBD product use.

1.0. Introduction

The last two decades have seen a dramatic shift in the availability, acceptability, and diversity of cannabis use methods in the United States. As of November 2020, forty-eight states (including Washington D.C.) had legalized cannabis consumption in some form, including thirty seven with comprehensive medical cannabis laws and sixteen with legalized recreational use (National Conference of State Legislatures: https://www.ncsl.org/)(National Conference of State Legislatures, 2018). Concurrently, perceived ease of access has increased among adolescents (Harpin et al., 2018), perceived harmfulness of cannabis has decreased among adolescents (Keyes et al., 2016), young adult disapproval of cannabis use has declined (Salas-Wright et al., 2016), public support for cannabis legalization has grown, and legislative changes have been associated with changes in cannabis attitudes, intentions, and norms (Clarke et al., 2018). The cannabis market has also increased in complexity as cannabis oil and flower vaporizers, commercial edibles, high-Cannabidiol (CBD)/low-Tetrahydrocannabinol (THC) cannabis, and other novel products grow in availability and popularity (Meacham et al., 2018; Miech et al., 2020). At the same time, an increasing number of cannabis products resemble tobacco products or facilitate cannabis and tobacco co-use (Giroud et al., 2015). Modern cannabis leaf vaporizers and vape pens frequently resemble e-cigarettes, and crossover products, such as pods contain THC for use in JUUL devices, facilitate co-use. It is likely that cannabis and tobacco co-use patterns are changing as the cannabis and tobacco markets co-evolve (Eggers et al., 2017). Survey measures that reflect changes and diversification in cannabis and co-use products (Hindocha and McClure, 2020) and behaviors can enable accurate and consistent surveillance to inform analyses of the health, social, and economic impacts of cannabis policy changes, reveal public health challenges as they emerge, and facilitate research on shifting perceptions, motivations, and consequences of cannabis and co-use.

The emergence of E-cigarette and Vaping associated Acute Lung Injury (EVALI) in the fall of 2019 provides an example case illustrating the benefit of up-to-date behavioral measures. Hospitalized patients had used many types of cannabis and nicotine vaporizers, and about half were co-users of cannabis and nicotine (Centers for Disease Control and Prevention (CDC), 2020). Surveillance data that separately measured oil and flower cannabis vaporizer use and accurately distinguished between cannabis and nicotine vaporizers would allow existing survey data to be used to investigate EVALI risk behaviors more accurately in the population.

Changes in the cannabis and co-use markets, behavioral patterns, and public perceptions have the potential to introduce sources of survey error into cannabis and co-use measures that must be investigated and addressed. Specification errors occur when a survey measure fails to correctly conceptualize the construct of interest (Johnson & Vangeest, 2017). For example, global questions about a substance that ignore specific forms of use have been reported to inadequately assess use (Johnson, 2014), and it is likely that the growing variety of cannabis products exacerbates this potential error. Specification error can also occur when measures employ terminology that differs from street or colloquial terminology. Measurement errors stem from factors that influence measurement quality, but are unrelated to the construct being measured (e.g. use of skip patterns, the mode of survey delivery, poor recall, and response editing) (Johnson, 2012, 2014). Response editing, or social desirability bias, occurs when a participant accurately recalls their substance use but underreports or denies use, often to conform to socially accepted behaviors (Krumpal, 2013), and particularly with illicit substances (Murphy and Rosenman, 2019) (although self-reported measures can be valid (Kedzior et al., 2006)). As legal status and public opinion changes, social desirability bias in cannabis measures might decrease.

Many types of research rely on survey measures of cannabis use, including behavioral studies, studies of the health effects of cannabis products, evaluation of cannabis policy, and toxicological studies that investigate exposure. Awareness of challenges for measuring cannabis will help researchers make informed choices about how to capture relevant patterns of use and avoid bias while maximizing accuracy and minimizing survey fatigue.

This study employs a mixed-methods approach to identify sources of potential specification and measurement error in survey measures of cannabis and cannabis-tobacco co-use (Fuchs, 2011), highlighting key measurement challenges and offering recommendations. We integrated an analysis of quantitative and qualitative reports to identify discrepancies and potential phenomena that might help to explain them, such as: inconsistent interpretation of the survey measures (specification error), interviews sparking memories (recall bias), and interviews soliciting additional disclosure (response editing).

2.0. Methods

This mixed-methods study compared self-reported cannabis use in qualitative interviews and quantitative surveys from 36 young adult cannabis and tobacco users in California. We supported and triangulated this analysis by simultaneously compiling a comprehensive dictionary of cannabis products and terms using extensive online searches. The integrated data collection and analytic strategies are illustrated in Figure 1.

Figure 1. Mixed Methodology.

Data were collected from participants (interviews and questionnaires) and cannabis websites and integrated in an interactive analytic process.

2.1. Data collection

This analysis was part of a longitudinal study of 60 poly-tobacco users age 18–29 in California designed to investigate patterns and practices of young-adult poly-tobacco use and the impact of marketing on these patterns. Young adults who used two or three of the following products – cigarettes, e-cigarettes, and smokeless tobacco – in the past 30 days were eligible for the study. Participants were recruited on Facebook and Craigslist and were largely from the San Francisco Bay Area, the Central Valley, and metro Southern California. At Wave 1 (January-August 2017), participants completed an online questionnaire about their past 30-day tobacco use and a semi-structured interview about their experiences with and routines of tobacco product use. Although past 30-day cannabis use was not an eligibility requirement, many participants described using cannabis in conjunction with their tobacco use during a Wave 1 interview. Based on this preliminary evidence, we formally incorporated cannabis use into our qualitative and quantitative data collection instruments in Wave 2 to conduct the present study.

We contacted participants by telephone, text message, and/or email to arrange a follow-up interview for Wave 2 (April-September 2018). Follow-up semi-structured interviews lasted 45–60 minutes and were conducted face-to-face in private university offices or over the phone. Interviews used open-ended questions to solicit description of participants’ experiences and routines of tobacco and cannabis and co-use. Interviews investigated the interplay between tobacco and cannabis use, including the extent to which the substances were intentionally used at the same time in the same delivery mechanism (i.e. co-administration) or to combine effects between products used within the same time frame (i.e. sequencing). Interviews were audio-recorded and professionally transcribed verbatim. Participants were compensated for their time. The University’s Institutional Review Board approved the study.

At Wave 2, participants completed a brief online questionnaire to capture demographic information and past 30-day use of tobacco and cannabis products a few days prior to the interview. The 36 participants who reported past 30-day cannabis use at Wave 2 comprise our sample. The survey measures covered both past 30-day use of cannabis alone and cannabis-tobacco co-administration (i.e. cannabis and tobacco used at the same time in the same product). Products included were: joint, spliff, blunt, pipe, bong or waterpipe, vaporizer with hash oil or concentrates, vaporizer with dried marijuana, dab rig, and edible cannabis products (Table 1). We created the measures as part of a survey development project led by the third author.

Table 1.

Survey questions administered to participants before semi-structured interview.

Preamble	The next questions are about the use of marijuana, which is also called cannabis. Marijuana is often used for fun or medical purposes, and people can smoke, eat, or vape it. When answering please include marijuana in all forms, including leaves and buds, powder/wax, glass/shatter, hashish, hash oil, resin, and all parts of the dried plant.
Past 30-day marijuana use	In the past 30 days have you used any type of marijuana? Yes, No
Past 30-day marijuana-tobacco co-use	In the past 30 days, have you used marijuana and tobacco or nicotine products together at the same time (e.g. in a spliff, blunt, or moke)? Yes, No
Number of days marijuana use, past 30	During the past 30 days, how many days did you use any type of marijuana? 0–30
Number of days marijuana-tobacco co-use, past 30	During the past 30 days, how many days have you used marijuana and tobacco or nicotine products together at the same time (e.g. in a spliff, blunt, or moke)? 0–30
Preamble	The next set of questions focuses on different ways that you may have used marijuana in the past 30 days, including joints, spliffs, blunts, pipes, bongs, vaporizers and dab rigs.
Preamble	A joint is rolled with marijuana only. A spliff is rolled with marijuana and tobacco mixed together.
PastS 30-day joint use	In the past 30 days, have you smoked a joint rolled with only marijuana? Yes, No, Prefer not to answer
Past 30-day spliff use	In the past 30 days, have you smoked a spliff rolled with marijuana and tobacco mixed together? Yes, No, Prefer not to answer
Preamble	Sometimes people take some tobacco out of a cigar and replace all or some of it with marijuana, or roll marijuana using a blunt wrap. This is sometimes called a “blunt.”
Past 30-day blunt use	In the past 30 days, have you smoked a blunt? Yes, No, Prefer not to answer
Past 30-day pipe use	In the past 30 days, have you smoked marijuana using a pipe? (Do NOT include bongs or waterpipes) Yes, No, Prefer not to answer
Past 30-day pipe co-use	When you smoked marijuana using a pipe in the past 30 days, did you ever mix tobacco in with the marijuana? Yes, No, Prefer not to answer
Past 30-day bong use	In the past 30 days, have you smoked marijuana using a bong or waterpipe? Yes, No, Prefer not to answer
Past 30-day bong co-use	When you smoked marijuana using a bong or waterpipe in the past 30 days, did you ever mix tobacco in with the marijuana? Yes, No, Prefer not to answer
Preamble	The next few questions are about vaporizers that are used to vape marijuana. These include larger table-top vaporizers (such as Volcano) and smaller, portable vaporizers such as e-joints or vaporizer pens, but do not include dab rigs.
Past 30-day oil vape use	In the past 30 days, have you used a vaporizer with hash oil or marijuana concentrates (such as hash, wax, resin, shatter, or budder)? Yes, No, Prefer not to answer
Past 30-day flower vape use	In the past 30 days, have you used a vaporizer with dried marijuana leaves, buds, or flowers? Yes, No, Prefer not to answer
Past 30-day vape co-use	In the past 30 days, have you used a vaporizer with marijuana and tobacco (dried loose tobacco or nicotine liquid) together? Yes, No, Prefer not to answer
Preamble	Next, we would like to ask about use of marijuana with a dab rig. Dab rigs are water-filled devices (similar to a bong) that often have a metal, glass, or quartz bowl (commonly called a “nail”) that is heated with a torch. Once the nail is very hot, the user touches marijuana concentrate to it using a stick or “dabber.” This produces a vapor that is inhaled through the dab rig’s mouthpiece.
Past 30-day dab rig use	In the past 30 days, have you used a dab rig with marijuana? Yes, No, Prefer not to answer
Past 30-day edible use	In the past 30 days, have you used edible marijuana (food, candy, or drink made with marijuana)? Yes, No, Prefer not to answer
Preferred co-use method	What is your preferred way of using marijuana and tobacco (dried loose tobacco or nicotine liquid) together? Spliff, Blunt, Pipe, Bong or waterpipe, Vaporizer; Smoking a cigarette, cigar, or cigarillo while high from smoking or vaping marijuana; Vaping nicotine while high from smoking or vaping marijuana, Some other way (describe), I do not have a favorite way, Prefer not to answer

We adapted existing measures assessing past 30-day marijuana use(Substance Abuse and Mental Health Services Administration, 2020) to create items that ask about past 30-day cannabis and co-use via various routes of administration. The items were further refined by conducting a pretest with n=300 participants who were asked to respond to each item followed by specific closed and open-ended questions to assess comprehension and recall. The pretest results were then used to determine the specific vernacular terms and range of response options that showed the highest comprehension and recall.

At the same time, we developed a comprehensive list of cannabis products, or a “cannabis dictionary.” To do so, we conducted a thorough search of online cannabis resources, including informational websites (e.g. www.leafly.com), community forums (e.g. Reddit.com), and cannabis dispensaries (e.g. www.eaze.com). We began by searching on each website for terms that were included in the questionnaire (e.g. vaporizer, dab rig). We identified and investigated new products/terms that these searches uncovered and determined we had reached saturation when emerging products/terms were used by only a small group of users and/or were specific to a geographic region.

2.2. Analytic methods

We used three integrated strategies to identify potential specification and measurement errors. First, we organized the results of the web searches into a typology of cannabis consumption practices (Figure 2) (Bowen, 2009). The development of this typology was iterative; during the interview process, we used the typology to revise the prompts in our interview guide to increase comprehensiveness of our questioning, and we used the findings of our qualitative analysis (see below) to revise and finalize the typology.

Figure 2.

A Cannabis Typology can be used to organize routes of administration, delivery methods, cannabinoids present, cannabis form, and strain type used in cannabis consumption.

We conducted two integrated analyses of the participant data: (1) we compared the qualitative and quantitative descriptions of cannabis use for each participant to identify discrepancies between the accounts and (2) conducted a qualitative thematic analysis of interview transcripts to develop a rich understanding of how products were described and used, to help interpret revealed discrepancies in the comparative analysis, and to reveal any new products or patterns that our survey measures did not capture. To do so, we developed a protocol to quantitize the cannabis behaviors described in the interview and a codebook. Two analysts (Author 1 and Author 2) conducted an initial in-depth reading of 3 transcripts, identified emergent codes, and independently used the participant’s interview transcript to complete the cannabis questionnaire (i.e. quantitizing the transcript data), blinded to each participant’s original questionnaire responses. We then discussed and resolved disagreements, developed a draft codebook, and developed a set of best practices for quantitizing the transcript. We then repeated this process with three additional transcripts (92.2% agreement on questionnaire responses), discussed areas of disagreement, revised the codebook and protocol, and conducted one final reading. Author 2 then coded remaining transcripts in Dedoose 8.0.42, a web-based qualitative analytic platform and completed the cannabis questionnaire during transcript coding based on interview content. After each transcript was coded and questionnaire completed, Author 2 compared her responses against participant’s responses to identify areas of discrepancy. She wrote a detailed memo outlining and providing possible justification for these discrepancies. Meanwhile, Author 1 led thematic analysis of coded excerpts, identifying key points for each case and making analytic notes on emerging patterns. Authors combined findings from these two analytic approaches into one table and identified patterns across cases. Working with the patterns identified through this integrated analysis, we derived the themes we present below. We reached data saturation, the point at which we were unable to identify new information, after reviewing 25 interviews. To confirm we had achieved saturation, Author 1 reviewed the remaining 11 transcripts and verified that further coding was unable to generate new insights.

3.0. Findings

The sample was racially/ethnically diverse (Latinx: 33.3% [n=12], White: 27.8%, Asian: 19.4%, Multi-racial: 16.7%; Black: 2.8%), had more men than women (66.7%), and was largely comprised of individuals who were attending or had attended some college (88.9%) (Table 2). Our analyses revealed several sources of potential specification and measurement error for cannabis and co-use measures, including non-differentiation of cannabis types, discordance in the interpretation of survey measures, and uncaptured variation in cannabis dosage within and between products. Notably, we found no evidence of survey recall bias.

Table 2.

Sample characteristics (n=36)

Age		24.2 years
		N (%)
Gender
	Female	12 (33.3)
	Male	24 (66.7)
Race/Ethnicity
	Latino/a/x/Hispanic	12 (33.3)
	White	10 (27.8)
	Asian	7 (19.4)
	Multiracial	6 (16.7)
	Black/African American	1 (2.8)
Education – at least some college		32 (88.9)
Past 30-day cannabis use
	Any cannabis	36 (100.0)
	Any co-administration	22 (61.1)
	Joint	25 (73.5)
	Pipe	19 (52.8)
	Oil vaporizer	23 (63.9)
	Edibles	19 (52.8)
	Flower vaporizer	9 (25.0)
	Spliff	10 (27.8)
	Blunt	16 (44.4)
Past 30-day binge drinking		31 (86.1)

^aCurrent cannabis use and binge drinking were measured by any occurrence in past 30 days

^bCurrent binge drinking was dichotomized from a continuous measure of the number of days the participant reported drinking at least 4 (women) or 5 (men) alcoholic shots or drinks within a few hours in the last 30 days.

3.1. Specification errors

3.1.1. Comprehensiveness

Using the cannabis dictionary and thematic analytic findings (Figure 1), we organized cannabis products into a typology with five dimensions: route of administration, delivery method, cannabis form, strain type, and major cannabinoids present (Figure 2). The typology focuses on features rather than products, improving the ability to uncover new products and/or methods of use. Route of administration describes how cannabis is consumed, including via inhalation (i.e. smoked and vaporized products), ingestion, and topical application. Delivery method depicts the physical contraption used to consume cannabis (or co-administer cannabis and tobacco), including vaporizers, pipes, and rolling papers. Cannabis form considers the specific consumable cannabis products, broadly categorized into cannabis flower and concentrates (e.g. hash oil, shatter, tinctures). Strain type refers to the vernacular strain of the cannabis plant – namely “Indica,” “Sativa,” or their hybrid). There is scientific debate over the taxonomic classification of these strains (Pollio, 2016) and evidence suggests the way these terms are used commercially does not reflect biological differences (McPartland, 2017; Piomelli and Russo, 2016). However, these strains have social meaning, as demonstrated by their widespread use in cannabis advertising and among cannabis users, and therefore might be relevant to social science studies of cannabis use perceptions and behaviors. Cannabinoids present refers to the presence and concentration of THC and CBD.

The integrated analysis illuminated the importance of differentiation (i.e. asking about products/forms/strains separately) because of the distinct contexts, purposes, and perceived effects across routes of administration, delivery methods, forms, cannabinoids, and strains. For example, the thematic analysis illustrated how cannabis with high CBD and low/no THC was used in different situations, for different purposes, and with different physical effects than cannabis with higher THC. For example, participants reported using CBD in situations where they wanted to maintain productivity, be functional while at work or school, or for health benefits without the psychoactive effects of THC, for example, “I have a topical and a tincture, and there’s no THC in that. So, I’m getting the great benefit of the plant without having to get high, which is nice.” High CBD (low THC) products were used to treat conditions such as aches and pain, menstrual cramps, nausea, endometriosis, anxiety, stress, sleeplessness, and headaches.¹

Like cannabinoids, cannabis strains served different purposes for participants. According to participants, Sativa strains yielded a “head high” and were often used during the day and evening. Indica yielded a “body high” and was often used at night as a sleep aid. For example, one participant explained the differences: “I like to have Sativa and Indica, so that I can like smoke Indica before I go to bed or something like that. Um, but I don’t generally like feeling sleepy when I smoke [cannabis]. Um, so [then] I tend to go for Sativa.”

Applying this typology to our survey measures revealed that the quantitative measures captured cannabis delivery methods but missed cannabinoids and strains.

3.2. Areas of discordance in product definitions

There were several instances of discordance in product definitions between researcher and participants. In interviews, several participants who reported using a CBD-only product in the interview, including vaporizers, tinctures, and topical cannabis, did not report that use on the survey (e.g., responding “no” to a global question about marijuana use). Our study included CBD-only products, so this misinterpretation was classified as under-reporting.

3.2.1. Co-use discordance

Definitions of co-use have varied in previous research; for example, some studies classify co-use as use of both cannabis and tobacco in the past 30 days regardless of overlap in day used or physiological effect (co-occurrence; (Hindocha and McClure, 2020)), while others specify only co-administration and/or sequencing.

On our questionnaire, three participants reported that they did not co-use cannabis and tobacco (answering “no” to a global question about co-use) but reported that they did use blunts. The thematic analysis of how all participants described blunts revealed why this might be the case. For example, several participants explained that the blunts they smoked did not have any tobacco mixed with the cannabis, which implied they were not a co-use product; as one participant with discordant co-use and blunt answers on the survey reported in her interview, “And then occasionally, probably once a week, I’ll smoke a blunt, but no tobacco in the blunt. We’ll remove all of the tobacco. And then you but the weed in it. And I never mix the two because it makes me [have a] headache. I would never smoke what they would call a spliff.” Other participants expressed uncertainty about the material of the blunt wrap, and so did not know that blunt wraps were made of tobacco leaf. This incongruence around blunts has been reported elsewhere and our study suggests it remains a measurement challenge (Lee, Battle, Lipton, & Soller, 2010).

Interviews also revealed a variety of co-use motivations, including co-use that was not intended to achieve compounding psychoactive affects through “boosting” (i.e. mixing cannabis and tobacco leaf to increase a high) or “chasing” (i.e. sequencing cannabis and tobacco consumption to increase a high). For others, use overlapped only because they were “chain-vaping” nicotine (i.e. using an e-cigarette continuously) and interspersing cannabis use: “My body has so much nicotine in it all the time that it doesn’t really change when I also smoke weed.”

When researchers and survey respondents interpret a question differently from each other, the data will then be misinterpreted by researchers. Misinterpretation can also result in inconsistent or underreporting, as was the case for CBD products, tinctures, and co-use in this study. If participants differ in their interpretation of a question (e.g. some participants report their CBD-only product use and others do not) the error is harder to quantify.

3.3. Measurement errors

3.3.1. Measurement validity

A majority of participants reported using more products on the survey than they discussed in the interview. For many, the additional products reported on the survey were used socially and provided by friends or coworkers. Participants reported both having preferred methods of cannabis delivery and an openness to consuming cannabis in other ways when offered in a social setting. There were also several cases in which participants remembered using products during the interview they had previously forgotten about, but no cases where participants disclosed product use that they appeared to have intentionally underreported on the survey. Many of these remembered products had not been used in the past 30 days or were not explicitly asked about on the survey (e.g. topical cannabis). It is unclear whether these participants would have reported using those products had they been asked about them directly. Overall, our analysis found very little evidence of recall bias or response editing, and strong evidence that measures of past 30-day use captured even infrequent use.

3.3.2. Skip Patterns

The discordance in product definitions we described above might also contribute to measurement error if these products are used with skip patterns. For example, if questions about individual co-administered products are skipped when someone answers “no” to a universal co-use question, some blunt users who do not report other co-use might not be shown a question about blunts.

3.3.3. Dosage

Thematic analysis revealed how cannabis dosage varied between and within delivery methods, between and within cannabis forms, and across contexts.

Dosage variation between and within delivery methods.

Participants illuminated how several factors contributed to dosage variation across consumption patterns. First, the strength and effect they experienced varied between products (e.g. edibles and dab rigs were often described as stronger than other means of consumption): “If you hit it from a dab rig, you’ll see people tank…from a pipe you’re going to smoke that same amount that that guy smoked in one hit, in like ten hits.” Other factors determining dosage for participants included: features of the delivery method (e.g. blunts vary in size based on the size of the original cigarillo, cigar, or blunt wrap), how products were used (e.g. smokers could control their high by changing how long they held in smoke before exhaling), and the ratio of cannabis and tobacco in co-administered products (which could, for example, depend on available supply of cannabis leaf/flower or preference during different times of the day). One participant described how they varied the ratio of cannabis to tobacco in a spliff across the day in order to yield different effects:

“The spliff I rolled in the morning was like 80 percent tobacco, or 90 percent tobacco, like almost a cigarette at that point, versus at the end of the day it’s probably 70 percent weed and 30 percent tobacco. And um, so during the day it’s sort of just like maybe get like a little buzz and sort of just chill…and at the end of the night it’s sort of like, yeah, I’m gonna sit here and watch Cartoon Network kind of thing.”

Dosage variation by cannabis form.

According to participants, dose also varied within and across cannabis forms. For example, the ratio of THC and CBD and the strain of cannabis affected potency and psychoactive and physiological effects. Vaping hash oil and vaping flower could also yield different effects. Comparing the two, one participant noted “I guess you could consider cannabis flower like a beer, and hash oil like taking shots.”

Context.

The context in which cannabis was consumed also related to the dose. A substantial amount of cannabis consumption occured socially. Cannabis products that were used at that time were often shared between individuals. A day of use reported on a survey measure could represent one hit from a shared joint, one or more whole joints smoked alone, or a level of consumption in between.

Dosage uncertainty.

Participant narratives also highlighted factors that contribute to their uncertainty about dosage. For example, social cannabis use could yield uncertainty for participants when the type of vaping device or strength of a shared product was unknown. One participant, a bartender, described a surprising high from a shared edible: “And the other day a friend gave me some weed-infused honey he made, and I made a little concoction and then took it as a shot. And Goddamn, that shit got me high as fuck…It’s like the tiniest bit; it’s not going to do anything…And I was just like, ‘oh crap.’ And I told the guy I was working with, ‘Dude, we need to close the bar really fast because I don’t know if I will be able to do it.’”

Several participants noted uncertainty about the strength and contents of products that were not regulated or were purchased on the illicit market, remarking that regulated products in legal markets were more trustworthy.

For participants, the ability to control psychoactive effects varied across delivery methods. Highs from edibles and tinctures were more difficult to titrate compared to smoking and vaping. One participant explained: “With edibles, you take it and then you have to wait. But with the vape pen it’s very immediate and it’s very incremental. So, you know in 30 seconds whether you want more or you’re okay.” Participants described the process of getting high with edibles as “trial and error,” “Russian roulette,” and “playing with like a gun.” Variations in THC content per serving, such as in homemade edibles, introduced substantial additional uncertainty about psychoactive effects.

These types of variation in cannabis use and co-use make it difficult to capture exposure in self-reported survey measures. These findings highlight the need for product differentiation, illustrate some limitations of survey measures, and suggest the need for creative strategies for new measures.

Units of measurement.

Participants used several units of measurement to describe consumption. For combustible and vaporized cannabis products, a “hit” was most commonly used. Participants would also “smoke a bowl” or other product, reference weight (“You would never find me with less than like 3.5 g – an eighth – on me, like ever”), and occasionally use how long a supply of cannabis would last as a unit of measurement (“I buy weed and it lasts a week or two for me, whereas some people just a few days”).

4.0. Discussion

We used a mixed-methods approach to identify potential sources of specification and measurement error with the goal of informing the development of accurate, comprehensive, and up-to-date survey measures of cannabis use. By combining survey and interview data from the same participants in California, this analysis was uniquely suited to investigate potential sources of specification error related to terminology and exclusion of new cannabis products and potential sources of measurement error from skip patterns, recall bias, and response editing. We developed a typology to conceptualize features of cannabis use and suggest using this typology to evaluate and maintain survey comprehensiveness as cannabis use grows increasingly diverse. Findings, implications, and suggested approaches are reported in Table 3.

Table 3.

Key Findings, Implications, and Suggested Approaches.

Key Findings	Measurement Implications	Suggested Approaches
Specification Error
* The cannabis market is changing quickly with new methods, devices, forms, and cannabinoid concentrations	- Out-of-date definitions - Misreported use	- Use detailed and updated definitions of marijuana/cannabis - Update product definitions as products change - Employ cannabis typology to capture multiple forms of cannabis consumption e.g. definition: The next question is about use of cannabis, which is also called marijuana. Cannabis is often used for fun or medical purposes, and people can smoke, eat, or vape it. When answering please include cannabis in all forms, including flower/bud, hashish, hash oil, tinctures, any food or drink containing cannabis (aka edibles), and products with CBD and no psychoactive effects.
Low/no THC products are used in different situations, for different purposes, and with different physical effects than high THC products	- Imprecise measurement of drug use - Imprecise measurement of THC exposure	- Differentiate between low/no CBD product use and other cannabis use - Explicitly ask about or exclude CBD-only products - Alternatively, differentiate between medicinal vs. recreational use e.g. When you use a vaporizer with hash oil or cannabis concentrates, do you typically use hash oil or concentrates that are: (1) CBD only (no THC) (2) Low THC / High CBD (3) Equal amounts of THC and CBD (4) High THC / Low CBD (5) I don’t know
* Inconsistent reporting of CBD-only products	- Measurement error of unknown magnitude	- Specify whether CBD-only products are included or excluded in cannabis definitions - Ask about CBD-only products separately from other cannabis e.g. When answering please include cannabis in all forms, including flower/bud, hashish, hash oil, tinctures, any food or drink containing cannabis (aka edibles), and products with CBD and no psychoactive effects.
* Blunts are not always considered co-use	- Underreported co-use (particularly when a global co-use question is used to screen and trigger skip logic) - Underreported blunt use	- Ask about blunt use to all marijuana users, including those who report no cannabis-tobacco co-use
Methods of co-use vary	- Underreported co-use that varies across method (e.g. underreports by users who are chain tobacco smokers/vapers who intersperse cannabis (unintentional overlapping))	- Specify methods of co-use e.g. co-administration: In the past 30 days, have you used cannabis and tobacco or nicotine together at the same time in the same product (e.g. in a spliff, blunt, or moke)?
Motivations for co-use vary	- Underreporting co-use if measures ask only about specific motives	- Ask about co-use methods and co-use motives separately - Specify which co-use motives are of interest (e.g. the item below asks about sequencing only) The next question is about using nicotine or tobacco after smoking or vaping cannabis. This is sometimes called “chasing.” In the past 30 days, how often did you smoke a cigarette, cigar, or cigarillo after smoking or vaping cannabis? (1) Every time I smoked/vaped cannabis (2) Some of the time (3) None of the time
Social use is common, introduces new ways of cannabis consumption, and affects dosage	- Social use/experimentation increases methods of exposure to drug and potentially to risks from the delivery method (e.g. trying dabbing) - Increases poly-cannabis use - Past 30-day measures include both regular (preferred) and social products - Product-specific measures of use will be noisy	- Differentiate between regular or preferred products and social/occasional products - Interpret measures with this finding in mind
Measurement Error
Response editing was low	- In this population, survey responses likely reflect use	- Work is needed to investigate in other contexts, such as where cannabis is not legal or more stigmatized
Dosage varies between and within devices and between and within cannabis forms	- Global measures of use will be noisy - Even product-specific measures of use will be noisy	- Differentiate between products - Specify CBD/THC content - Specify which cannabis form is used in vaporizers - Use THC biomarkers to enhance measures of exposure
Dosage can be uncertain; Product knowledge can be uncertain (e.g. about cannabinoids and strains)	- Respondents may have difficulty answering detailed questions - Difficulty may be higher for individuals who share with friends, purchase on the black market, or have less domain knowledge	- Tailor questions to the level of expertise of the target population (e.g. questions of the general population might be less detailed than questions of daily cannabis users; questions in states without legal cannabis might be less detailed than questions in states with legal cannabis) - Include “I don’t know” option for CBD/THC concentration and dosage

^*indicates findings we encourage every survey researcher to address in their measures

For each source of potential specification or measurement error, we describe the primary implications for survey measurement and potential solutions. Survey researchers must balance maximizing accurate measurement and minimizing survey fatigue by prioritizing the needs of their research question.

4.1. Specification error

The typology we developed illustrated how different cannabis properties (e.g., strain, cannabinoids) might impact cannabis perceived psychoactive effects, patterns of use, and reasons for use. Future studies to understand how and why people use cannabis might include questions about strains or cannabinoids in survey measures, as motivations for use may vary significantly across these dimensions, and might be missed by a generic measure of “cannabis use.” Similarly, life science studies to determine the biological effects of cannabis use might also consider measuring differences in cannabis strain or cannabinoids in controlled studies to start to address whether perceived differences in cannabis products actually impact biological outcomes.

Survey responses depend on whether participants consider CBD products to be marijuana and blunts to be co-use. Surveys that use global questions to assess overall use and then apply skip logic based on the first response might eliminate individuals who use the products of interest but do not consider them to be relevant (e.g. blunt users who do not consider using blunts to be co-use; CBD vape users who do not consider them to be marijuana) (resulting in measurement error). We recommend that researchers be explicit about whether or not cannabis measures include CBD-only (or high-CBD) products such as vaporizers, edibles, tinctures, and topicals and generally avoid skip logic that relies on terms that have multiple meanings or that might result in underestimated use.

4.2. Measurement error

Survey measures capture frequency of cannabis use but often do not capture quantity (usual, maximum, and minimum dose/style of use per occasion) (Day and Robles, 1989). For our participants, this last parameter – quantity – was highly variable across products, within products, and across contexts, consistent with previous research comparing cannabis content and physiological effects between products (Cooper and Haney, 2009; Mariani et al., 2011). Previous work has also demonstrated that edibles have high uncertainty in dosing (Giombi et al., 2018; Lamy et al., 2016; Popova et al., 2017).

In combination with previous literature, our findings suggest that product differentiation (i.e. asking about products separately) can improve the accuracy of self-reported cannabis use. However, differentiation does not address dosage variation within products or cannabis forms. For example, a bong can contain only cannabis or it might include tobacco, it can be smoked alone or shared with friends, and it can be smoked in one sitting or over the course of a day. This variety in dosage makes it hard to quantify consumption using existing self-reported survey measures. Previous work that has investigated between-product variation in dosage (e.g. joints vs blunts) (Cooper and Haney, 2009; Mariani et al., 2011) should be extended to within-product comparisons to identify the range of possible exposures across common uses of each single product, examine differences in those distributions across products, and identify how survey measures can effectively and efficiently capture this last parameter of use.

In light of the complexity of use behaviors and inconsistent relationship between product use and exposure to cannabinoids, survey measures could be coupled with biomarkers of exposure for studies that seek to classify use or co-use and its intensity (Huestis, 2009). Presently, cannabis abstinence can be confirmed via urine analysis, and monitoring of 11-nor-Δ⁹-tetrahydrocannabinol-9-carboxylic acid (THC-COOH) can help identify recent or frequent THC exposure (Musshoff and Madea, 2006). A recent study presented a method to biochemically verify CBD exposure (Goggin and Janis, 2020). As both the precision of biomarkers and the social acceptability of cannabis use increase (which could make sampling easier), biomarkers may provide valuable information that complements self-reported survey measures.

4.3. Recall bias and response editing

Our analysis suggested that recall bias and response editing of past 30-day use was limited. Limited recall bias might be because of the relatively short time-frame of our inquiry; previous research found higher underreporting when substance use was assessed using wider time frames such as past-year or lifetime-incidence (Bachman and O’Malley, 1981; Fendrich and Vaughn, 1994; Johnson et al., 1998). Cannabis is very commonly used in social settings, particularly among light or experimental uses (Phillips et al., 2018), and our findings suggest that social exchange frequently introduced non-routine products to participants; participants expressed the most uncertainty about vaping device, content, and dose for social or shared products, which might impact the accuracy of responses. Survey measures might allow participants to indicate if use of a product or form is primarily shared/borrowed or purchased/owned as a rough indicator of both dose and quality of self-report. The difference between routine products and social products also suggests that survey measures asking about “past 30-day use” of products will collect more information than measures about “regular use” of products.

Developing more comprehensive measures of cannabis can add nuance to existing behavioral and policy research. Given the variety of cannabis consumption methods, social contexts, and perceived meanings (Soller and Lee, 2010), detailed cannabis measures in health surveys would allow future work to examine whether policy effects differ by delivery methods, cannabis forms, and cannabinoids. As described in this manuscript, co-use occurs in a variety of ways including co-administration, closely timed use of separate delivery methods to enhance the effects of one or both substances (sequencing), and simultaneous use without intention to enhance the effects of one or both substances (unintentional overlapping). Clear measures of co-use and changes in co-use patterns, especially those that can differentiate between co-administration, sequencing, unintentional overlapping, and general co-occurrence, would enhance ability to study the interplay between cannabis and tobacco products (Hindocha and McClure, 2020; Ramo et al., 2012). The news coverage reporting cannabis oil vaporizers as a likely cause of EVALI may impact both cannabis and nicotine product use behavior. Cannabis legalization might also influence co-use and tobacco use; cannabis, co-use, and cannabis smoke have been perceived as safer than tobacco use (Keyhani et al., 2018; McDonald et al., 2016; Schauer et al., 2017). As legal penalties for cannabis, the largest perceived risk of cannabis use (Keyhani et al., 2018; Roditis and Halpern-Felsher, 2015), are removed, those who perceive cannabis to be less harmful than tobacco might take up or increase cannabis use, and patterns may differ across cannabis products.

This study suggests several important features of survey measures for cannabis use, broadly: clarity about what devices and forms are included in each question, product differentiation rather than a single “cannabis use” question, explicit distinction between (or explicit exclusion of) high or only-CBD cannabis use, and improved measures of dosage. The best strategy for addressing these measurement challenges will depend on the research question. For example, a study of psychoactive drug use might want to explicitly exclude CBD products, while a study of the effects of medical cannabis laws might include them. Furthermore, no single questionnaire can capture all of these dimensions without increasing survey fatigue, and researchers will have to prioritize. For example, a study of the effects of medical cannabis legalization might differentiate between low and high THC content to study motives, whereas a study of the effects of cannabis legalization on tobacco use might prioritize detailed measures of co-use. A toxicology study might require precise measures of dosage, whereas a study of cannabis use prevalence might not.

4.4. Limitations

This study was conducted in California, with many participants from the San Francisco Bay Area, where the sale of recreational and medicinal cannabis is legal, cannabis use may be uniquely normalized, and the cannabis market uniquely sophisticated. We do not know whether we would reach similar conclusions with participants in different policy or cultural contexts. All of the participants in our sample were poly-tobacco users when recruited; while they were uniquely suited to provide detailed responses about their cannabis and cannabis-tobacco use, their experience might not reflect that of the broader population of cannabis users. Our thorough web search mitigated any regional effects in identifying products, terms, and typological dimensions.

This study did not recruit poly-tobacco users based on cigar use, and therefore it might not fully reflect the experience of cigar users. However, 10 of the 36 participants reported past 30-day cigar use at Wave 2, while 6 reported past 30-day smokeless tobacco use.

Participants in our study might have underreported cannabis use, particularly frequency or locations of use (Johnson, 2014), reflecting drug use stigma and social desirability bias. The interview setting, most frequently face-to-face interviews in a research center at a major university (Aquilino & Sciuto, 1990; Currivan, Nyman, Turner, & Biener, 2004; Johnson, 2014) might have discouraged substance use reporting. Our location in a legal state and explicit study focus on substance use might mitigate these effects, and we expect any underreporting to be modest. Underreporting the frequency of use is unlikely to impact our results. Self-reported measures depend on participant knowledge about their product use, for example, the concentration of THC and CBD in the products they consume; we cannot address limitations in participant knowledge.

5.0. Conclusion

Our analysis illustrates how the complexity of cannabis use can bias survey data and limit research results and offers suggestions to survey researchers on how to address these challenges. It is important to collect surveillance and monitoring data on cannabis use as the policy environment changes and we learn more about the relative health effects of different cannabis products. This requires up-to-date and nuanced reporting of cannabis use behavior.

Highlights.

• Integrated mixed-methods analysis illuminates potential survey error in cannabis measures.

• Measurement threats include inconsistency in cannabinoid content, dosage, and interpretation.

• No evidence of survey recall bias or response editing was detected.

• A typology of cannabis products and suggested strategies for survey measures are provided.