Evaluation of Availability of Survey Data About Cannabis Use

Key Points

Question

Are US national data measuring aspects of cannabis use consistently available across states and over time?

Findings

This case series study examining 7 state and US nationwide surveys found that limited data are available at the state or national level to monitor 8 key dimensions of cannabis use and perceptions.

Meaning

These findings suggest that despite growing public support for cannabis legalization, monitoring public health effects of state and national policies may be difficult owing to a lack of available data in existing population surveys.

This case series study examines the types of survey data available on cannabis use among aming US adults and adolescents.

Abstract

Importance

In response to increasing public support for cannabis legalization, understanding associations of state and federal policy changes related to cannabis legalization with patterns of cannabis use is important. A challenge for public health monitoring and research is significant variation in data availability related to cannabis use behaviors and perceptions across and within states and over time, including the availability of prelegalization vs postlegalization data.

Objective

To review data available on cannabis use and related behaviors over time in Massachusetts and the US.

Design, Setting, and Participants

This case series examined state and national surveys on public health and related behaviors and outcomes to review availability of cannabis-related data for Massachusetts for 8 key indicators over time. Additionally, the Behavioral Risk Factor Surveillance System (BRFSS) and Youth Risk Behavior Surveillance System for all states were reviewed. The analysis was conducted between February 1, 2019, and March 18, 2020.

Exposures

Surveys that enable state-level estimation of cannabis use and related behaviors.

Main Outcomes and Measures

Eight key indicators related to cannabis use behaviors and perceptions: lifetime cannabis use, age of initiation, frequency of use, location of use, method of use, source of cannabis, perceptions of cannabis, and reason for use (ie, medical vs nonmedical).

Results

There were 7 surveys that monitored cannabis use and related behaviors in Massachusetts for adolescents and adults. No surveys monitored all 8 indicators of interest, and availability over time was limited. In the most recent BRFSS, 24 states asked cannabis-related questions, meaning BRFSS data on cannabis use was lacking for more than half of the US adult population. In the Youth Risk Behavior Surveillance System, 36 states asked standard cannabis questions; most other states had at least 1 question related to cannabis use and frequency.

Conclusions and Relevance

These findings of limitations of existing surveys, and particularly the lack of national questions in the BRFSS and Youth Risk Behavior Surveillance System, suggest that available data have substantial limitations for monitoring cannabis use. As cannabis policy changes continue, there is a need to remain focused on the availability of high-quality data sources that allow for critical public health research.

Introduction

As of December 2019, 11 states and the District of Columbia (DC) have legalized cannabis for adult nonmedical use¹ and 33 states and DC have legalized cannabis for medical purposes.² In response to growing public support³ for cannabis legalization, understanding the effects of state and federal policy changes related to cannabis legalization on patterns of cannabis use is important.⁴ There is intense policy interest, including by regulators, in monitoring a number of indicators associated with cannabis use behaviors and perceptions among adolescents and adults to detect, and if necessary reduce, any negative public health effects of cannabis legalization.^5,6,7 A challenge for monitoring and research is significant variation in data availability across and within states and over time,^4,7,8,9 including availability of prelegalization vs postlegalization data.

Variation in available data across states highlights the need to better understand the extent of existing data in measuring effects of state and federal policy changes associated with cannabis legalization. Given ongoing policy and regulatory changes (eg, retail markets, packaging, labeling),^5,7,10 it is critical to understand whether data are available to monitor a number of cannabis use indicators associated with specific legislative or regulatory provisions. These provisions continue to evolve as states implement legalization with varying policy and regulatory structures. Specific cannabis-related behaviors of importance include frequency, location, and methods of use; these behaviors may be risk factors for problematic cannabis use and associated outcomes.^{11,12,13,14,15} Understanding data availability to monitor cannabis use in nationally comparable surveys is critical to be able to compare findings from different states,^7,9,16,17,18

Many states with legal cannabis have statutorily required research agendas to monitor outcomes, including public health.^9,19,20 Negative consequences associated with cannabis use among youth are well documented and include neurocognitive defects, mental health issues, risk for dependence, lower educational attainment,^21,22,23 and e-cigarette or vaping product use–associated lung injury.^24,25,26 Methods of cannabis use, including among youth, are changing. A decreasing number of individuals who use cannabis report smoking, and an increasing number report use of nonsmoking modes, such as edibles, vaping, and dabbing,^{27,28,29,30,31} for which little is known about possible adverse health effects.

There is conflicting evidence about whether the prevalence of recent (ie, within the past 30 days) cannabis use increases after legalization,^16,29,32,33 but it is clear that prevalence and frequency of use have increased significantly over the past 2 decades.^34,35 In 2018, 45% of individuals older than 12 years had ever used marijuana and 16% used marijuana in the past year.³⁶ Perceptions of risk associated with cannabis use have decreased, particularly among youth, which has been associated with a higher likelihood of future use.^3,33,37,38

To better understand available data to monitor cannabis use among adolescents and adults, we conducted a scoping review and evaluation of survey data over time to monitor 8 indicators related to cannabis use and perceptions in Massachusetts.³⁹ We also sought to inform future research and monitoring efforts by evaluating the utility of national data sources for generating state-level estimates of cannabis use and perceptions that can be compared across US states. We hypothesized that state-level data would better capture the 8 indicators over time than national data.

Methods

We used a case series approach to identify and examine data sources available to measure cannabis use. For Massachusetts, our in-depth case study site, we analyzed data availability over time and assessed specific cannabis use questions for the general adult population as well as specific at-risk populations, including adolescents. We focused on Massachusetts because the legislation authorizing the legal retail cannabis market outlined annual monitoring requirements, but the extent of data availability to meet those requirements was unknown.³⁹ We then complemented this in-depth assessment of Massachusetts data with a review of cannabis use questions in major national data sources as they apply to all 50 states and Washington, DC. The analysis was conducted between February 1, 2019, and March 18, 2020. This study was determined to be not human participants research by the University of Massachusetts Human Research Protection Office. We followed the reporting guideline for case series.⁴⁰

Data

We used publicly available documents describing national and state-specific data sources to determine cannabis use data availability. We first conducted a scoping review in Massachusetts to determine available data sources to measure cannabis use at the state level for the general population and specifically for adolescents. We identified data sources using published literature, national and state government surveys, and key informant interviews with state government agencies’ staff. We used website and document review to determine specific survey questions and inclusion of specific questions over time. We used the most recent questionnaire available online (as of April 15, 2019), unless the most recent version available was administered before 2015, in which case we contacted the data provider for more information. We limited this analysis to survey data that could identify prevalence of cannabis use and/or related behaviors of interest at the state level. We excluded national surveys that could not produce state-level estimates. We also excluded surveys that captured only local (ie, within-state) estimates.

We analyzed data sets available at the national level in more detail. The Behavioral Risk Factor Surveillance System (BRFSS) is jointly conducted by the Centers for Disease Control and Prevention and state agencies annually and is designed to be representative of the noninstitutionalized adult population in a state. We first determined which states used the national marijuana module containing questions related to cannabis use, frequency of use, method of use (eg, smoking, edibles), and medical vs nonmedical use. For states that did not use the national marijuana module in the most recent year for which this information is systematically available from the Centers for Disease Control and Prevention (ie, 2017), we examined state BRFSS websites to review the most recent questionnaire (as of September 3, 2019). If the questionnaire was older than 2015, we contacted the state agency responsible for the most recent version. For the Youth Risk Behavior Surveillance Survey (YRBSS), we examined state YRBSS websites to review the most recent questionnaire (as of March 15, 2020); if the questionnaire was not available, we used the most recent results available from the state YRBSS website or the Centers for Disease Control and Prevention website.

Statistical Analysis

We assessed which data were available in Massachusetts and recorded years available, frequency of collection, population representation, age range, number of individuals surveyed, and access information. For each data source that met the inclusion criteria, we used survey questionnaires from 2011 to the most recent year available to determine cannabis-related questions over time. We analyze 8 key indicators of cannabis use patterns and perception of interest to the Massachusetts legislature, as codified in the statute that legalized cannabis for adult use in 2017¹⁹; this is a robust set of indicators of interest to public health researchers and policy makers. These 8 indicators include lifetime cannabis use, age of initiation, frequency of use, location of use, method of use, source of cannabis, perceptions of cannabis, and reason for use (ie, medical vs nonmedical).

For the BRFSS and YRBSS (including high school students only), after the collection of the questionnaires, we analyzed inclusion of cannabis-related questions across states. For BRFSS, we categorized states into 4 categories reflecting whether the state (1) used the national marijuana module, (2) included state-specific questions that were at least equivalent to the national marijuana module, (3) included state-specific questions related to cannabis but did not include all questions from the national module, or (4) did not include any cannabis-related questions. We used state population estimates from the 2018 bridged-race population estimates⁴¹ of adults aged 20 years and older to determine national BRFSS coverage with questions related to cannabis use. For the YRBSS, we categorized states into 4 categories reflecting whether the state (1) used the marijuana questions included in the national or standard questionnaire, (2) asked at a minimum about marijuana use and frequency of use in the past 30 days, (3) other, or (4) did not conduct the YRBSS. Questions from the BRFSS marijuana module and national or standard questions from the YRBSS questionnaire are presented in eTable 1 in the Supplement. To provide further context, we examined state policies and categorized them on the basis of the legal status of cannabis (ie, legalized general adult use, legalized medical use, decriminalized, or fully illegal) as of December 2019. Maps were created using ArcMap software version 10.7.1 (Esri).

Results

We found 7 surveys containing cannabis use information for Massachusetts residents, including national and state surveys (Table 1). We included the national BRFSS for comparison purposes, although it cannot produce estimates for cannabis use in Massachusetts. Of these 7 surveys, 3 are national surveys that can produce state-level estimates and 4 are Massachusetts-specific surveys. There is some overlap in these surveys, as the National YRBSS and Massachusetts Youth Risk Behavior Survey (MYRBS) are conducted simultaneously. Overall, these surveys represent a comprehensive age range, with several adult-specific surveys, several adolescent-specific surveys, plus the National Survey on Drug Use and Health (NSDUH), which includes information on individuals 12 years and older, and the National Health and Nutrition Examination Survey (NHANES), which includes information on all ages.

Table 1. Summary of Survey Data Available to Monitor Cannabis Use Patterns, Perceptions, and Methods of Consumption.

Data name	Source	Population included	Age range, y	Individuals, No.	Years (frequency)	Access (cost)
National data with MA indicator
BRFSS	CDC	Civilian, noninstitutionalized US residents	≥18	400 000/y	1984-2017 (annually)	Public use (free)
NHANES	CDC	Civilian, noninstitutionalized US residents	All	5000/y	1999-2016 (annually, with 2 y panels)	Public use (free)a
NSDUH	Substance Abuse and Mental Health Services Administration	Civilian, noninstitutionalized US residents	≥12	65 000/y	1971-2017 (annually)	Public use (free)a
YRBSS	CDC	Middle and high school studentsd	11-18	15 000/y	1990-2017 (odd-numbered years)	Public use (free)b
MA-only data
MA BRFSS	CDC and MA DPH	Civilian, noninstitutionalized MA residents	≥18	7000/y	1984-2017 (annually)	Limited use (free)c
MA Marijuana Baseline Health Study	MA DPH	Noninstitutionalized MA residents	≥18	3000	2017 (one time)	No information available
MYHS	MA DPH and DESE	MA middle and high school students	11-18	5500/y	2007-2017 (odd numbered years)	Limited use (free)c
MYRBS	MA DESE and CDC	MA high school students	13-18	3300/y	2007-2017 (odd numbered years)	Limited use (free)c

Abbreviations: BRFSS, Behavioral Risk Factor Surveillance System; CDC, Centers for Disease Control and Prevention; DESE, Department of Elementary and Secondary Education; DPH, Department of Public Health; MA, Massachusetts; MYHS, Massachusetts Youth Health Survey; MYRBS, Massachusetts Youth Risk Behavior Survey; NHANES, National Health and Nutrition Examination Survey; NSDUH, National Survey on Drug Use and Health; YRBSS, National Youth Risk Behavior Surveillance Survey.

^aLimited use at Research Data Center required for state indicators and drug use questions for children.

^bState level data are available directly from the CDC in 37 states; 10 states require obtaining data directly from state agencies conducting the survey through a variety of application processes.

^cRequires application process.

^dOnly includes students in MA high schools.

Our review of detailed survey information to monitor cannabis use over time found that these surveys capture a wide variety of items and time periods, but no survey captures all 8 indicators of interest simultaneously (Table 2). The NHANES and the NSDUH include the most information for adolescents and adults, as they capture time series information about lifetime use, age of initiation, and frequency of use using standard questionnaires administered nationally. The NSDUH also captures information over time related to the source of marijuana, perceptions of marijuana, and reason for use. Although the national BRFSS marijuana module captures some of this information systematically, Massachusetts and 38 other states do not use this module.

Table 2. Availability of Survey Data to Monitor Cannabis Use by Data Source.

Data name	Lifetime use	Age of initiation	Marijuana use			Source of marijuana	Perceptions of marijuana	Medical vs nonmedical use
			Frequency	Location	Method
National data with MA indicator
BRFSS	NA	NA	2016-2018a,b	NA	2016-2018a	NA	NA	2016-2018a
NHANES	2011-2016	2011-2016	2011-2016	NA	NA	NA	NA	NA
NSDUH	2011-2017	2011-2017	2011-2017	NA	NA	2011-2017	2011-2017	2013-2017
YRBSSc	2011-2017	2011-2017	2011-2017	2011	2015	NA	NA	NA
MA-only data
Massachusetts BRFSSd	2015-2017e	NA	NA	NA	NA	NA	NA	2015-2017
MA Marijuana Baseline Health Study	NA	NA	2017	NA	2017	NA	2017	2017
MYHS	2011-2017	2011-2017	NA	NA	NA	NA	2011-2017	NA
MYRBSf	2013-2017	2013-2017	2013-2017	2013-2017	NA	NA	2017	NA

Abbreviations: BRFSS, Behavioral Risk Factor Surveillance System; MA, Massachusetts; MYHS, Massachusetts Youth Health Survey; MYRBS, Massachusetts Youth Risk Behavior Survey; NA, not available; NHANES, National Health and Nutrition Examination Survey; NSDUH, National Survey on Drug Use and Health; YRBSS, National Youth Risk Behavior Surveillance Survey.

^aIncluded in an optional module.

^bMA does not use this module.

^cAvailability over time is based on questions available in the national and standard questionnaires.

^dIncludes questions about marijuana use over time but does not ask questions consistently outside of the noted periods.

^eDoes not includes questions explicitly about lifetime marijuana use but asks about use in past year.

^fIncludes questions about problematic use but not about frequency of use.

State-specific survey data are available to fill some information gaps, although these surveys are not available over a long period, particularly in the prelegalization or premedical cannabis periods. The Massachusetts BRFSS includes questions related to problematic cannabis use; however, there were a number of changes in the questions related to cannabis over time, which makes it difficult to examine changes in behavior over time (eTable 2 in the Supplement). The Massachusetts Marijuana Baseline Health Study, conducted in 2017, captured baseline frequency of use, method of use, perceptions of marijuana, and reason for use,⁴² similar to the information captured by the national BRFSS module; however, it is unknown to what extent these measures will be assessed over time, as Massachusetts state agencies have not publicly disclosed plans to repeat this survey.

The BRFSS is a powerful national survey owing to its size, time scale, and the frequency of administration, which may allow for within- and across-state estimates; we analyzed this survey to determine which states consistently capture cannabis use data (Figure; eFigure in the Supplement). Of 50 states and DC, we found that 11 states used the national marijuana module as of 2017 (plus 1 additional state in 2018), 4 states did not use the national marijuana module but asked questions that were at least equivalent to the national module, 9 states asked any state-specific marijuana questions, and 27 states did not ask any marijuana questions (Table 3). Using state population estimates, we found that 32.2% of the US population is covered by the national marijuana module or equivalent state-specific questions.

Figure. Cannabis Questions Available in National Surveys.

A, The most recent questionnaire available used to determine questions. Alaska and Hawaii not shown; Alaska used the national marijuana module, and Hawaii did not have cannabis-related questions. Ohio added the national marijuana module in 2018. B, Other includes the Massachusetts questionnaire, which includes a question related to 30-day use of marijuana, but does not include 30-day frequency of use; 7 states (ie, Alabama, Georgia, Indiana, Mississippi, New Jersey, Ohio, South Dakota) did not achieve adequate response rates to report population weighted estimates in 2017.

Table 3. Summary of Behavioral Risk Factor Surveillance System State-Level Use of Cannabis Related Questions.

Category of questions	National population covered, %	States, No.a
		Overall	Legalized		Decriminalized	Fully illegal
			General adult use	Medical useb
National Marijuana Module	26.6	11	2	3	1	5
Equivalent state marijuana questions	5.6	4	1	3	0	0
Any state marijuana questions	11.5	9	6	2	0	1
No marijuana questions	56.3	27	3	13	2	9

^aStates were classified by the most permissive aspect of their cannabis policy.

^bLegal medical use includes the use and/or sales of at least some product or plant matter containing delta-9-tetrahydrocannabinol. Seven states with legal medical cannabis have not decriminalized possession of small amounts (ie, Arkansas, Louisiana, Montana, New Jersey, Oklahoma, Pennsylvania, Utah).

For adolescents, the YRBSS, MYRBS, and Massachusetts Youth Health Survey are conducted among Massachusetts high school students (and middle school students for the Massachusetts Youth Health Survey). These are conducted in public schools every 2 years and representative of a large but select group of adolescents. These surveys capture information about lifetime use, age of initiation, frequency of use (not among middle school students), perceptions of marijuana, and location of use specific to school property. Intermittent information is available on method of use, but this has not been captured systematically over time. We analyzed questions in YRBSS state questionnaires and found that 36 states ask questions consistent with the standard or national questions in the most recent year (Figure, B; eFigure in the Supplement). Substantial variation exists in state level questionnaires, with 6 states asking questions related to mode of use, 6 states asking questions related to perceptions of use, and 3 states asking about use on school property, among others. In 2017, 7 states did not have adequate response rates to report population-weighted estimates.

Discussion

This case series study found a number of sources that could be used to monitor cannabis use over time, but many of these data sources have substantial limitations that restrict their usefulness in understanding effects of state and federal policy changes. Our results highlight gaps in state data collection and how these gaps affect knowledge surrounding the public health effects of cannabis legalization.

Contrary to our hypothesis, we found that Massachusetts—which legalized medical cannabis in 2012 and general adult use of cannabis in 2016—does not have consistently available state data sources to monitor all 8 key cannabis use indicators of interest and that state-level data were not as robust as national data sources. Despite the public health importance of maintaining nationally comparable survey data over time to allow for outcomes analyses of anticipated and unanticipated policy changes,⁹ our results show this is not happening consistently within Massachusetts or using the BRFSS and YRBSS nationally. We found that although 2 major national surveys, the NSDUH and NHANES, can provide substantial information about multiple dimensions of cannabis use over time, these surveys are limited by sample size in their ability to make across- and within-state comparisons, particularly at the annual level. This is an important limitation when one considers the rapid pace at which legalization and related policies are changing and the need to detect potential changes in key indicators in as timely a manner as possible.

Monitoring cannabis use among adolescents is of particular public health importance owing to adverse effects associated with cannabis use on brain development, potential substance use dependence later in life, and changes in perceptions around cannabis use and changing methods of use.^{3,27,28,29,30,31,33,37,38} In Massachusetts, we found that there are national and state-specific surveys available (under restricted access policies from data providers) that monitor cannabis use and frequency; however, these surveys have not consistently collected information over time about method of use, location of use, and/or perceptions of cannabis, all of which are important to fully quantify changing patterns of use over time. As cannabis is legalized for adult use, monitoring diversion of legal cannabis into the adolescent population is particularly important. These youth surveys, available in Massachusetts, are not available in all states with medical or nonmedical cannabis.⁹ Colorado is a leader in carefully tracking these measures. Colorado was one of the first states to legalize adult use of cannabis and has carefully tracked the public health implications of legalization, including through the addition of questions over time to the Healthy Kids Colorado Survey, such that the most recent version included questions on 6 of the 8 dimensions we examine (ie, lifetime use, age of initiation, location of use, method of use, source of marijuana, and perceptions of marijuana).⁴³

Understanding the availability and comparability of BRFSS data across states with respect to cannabis-related questions is a novel contribution of our analysis. The NSDUH is able to make state-specific estimates at the annual level using restricted data or using 2-year periods using public use data; additional national data sources are necessary to research the associations of time-sensitive state policy changes and make within-state comparisons. The BRFSS has a much larger sample size at the state level, but we found that a large number of states, with more than 56.3% of the national population, do not include any cannabis use questions. The national marijuana module became available for use in 2016; in early legalization states (ie, states that legalized cannabis before 2016), the national module does not contribute substantial information for understanding changes in cannabis use during longer periods or across the prelegalization to postlegalization transition. However, if implemented in states prior to legalization for adult use, the national BRFSS marijuana module would likely provide useful data.^3,9 This shows the importance of changes to BRFSS core questions, including adding questions on cannabis use, in a rapidly changing policy environment.^3,4 This is particularly important to monitor outcomes among at-risk population groups, such as adolescents, persons with substance use or mental health disorders, and pregnant women, who may be disproportionately affected by cannabis legalization and for whom there is likely not adequate sample size to examine carefully at the state level in national surveys. Additionally, to our knowledge, no data are available to measure cannabis use among incarcerated populations, who have high rates of substance use disorders.⁴⁴

Limitations

Our study has several limitations. The first limitation is that our primary analysis of available surveys was limited to Massachusetts; as we show with variation in availability in the BRFSS and YRBSS, substantial differences in data availability exist across states, so data available in Massachusetts may not be generalizable to the rest of the country. However, this in-depth analysis of data available for Massachusetts is an important case study to understand what may be available in other states and reveals issues likely to be common across states. The second limitation is that we limited our analysis to survey data, as this is the most commonly used way of obtaining prevalence estimates for cannabis use.⁹ Administrative or other data sources, such as health insurance claims data or substance use treatment facility discharge data,⁴⁵ may also capture changes in cannabis use associated with cannabis use disorder. Owing to the very small number of surveys measuring polysubstance use, we were not able to consider concurrent use of cannabis and other substances, such as alcohol. This is an important extension in considering future data collection, particularly since little is known about polysubstance use for adults and adolescents.^46,47 Third, because we included only data sources that support estimation at the state level, we excluded a number of data sources with potential relevance to research on cannabis policy (eg, the American College Health Survey and Monitoring the Future). Fourth, our focus on the general population omitted some important subgroups, such as pregnant women, for whom there is growing interest and concern regarding cannabis use.

Conclusions

In this case series study, we report 2 large, high quality, nationally representative surveys—NHANES and NSDUH—that can be used to monitor cannabis use and the outcomes associated with state and national policy changes; however, substantial challenges remain in the availability of data needed to fully monitor the public health effects associated with cannabis legalization. Existing surveys, and particularly the lack of national core questions in the BRFSS or standardized required questions in the YRBSS, suggest that available data have substantial limitations for monitoring cannabis use along the 8 dimensions we examined. As research into the effects of cannabis legalization and other changes in cannabis policy continue, remaining focused on the availability of high-quality data sources will allow critical public health research.

Supplement.

eFigure. Cannabis Questions Available in the Behavioral Risk Factor Surveillance System and Youth Risk Behavior Surveillance Survey and State Cannabis Legalization Status

eTable 1. Cannabis Use Questions in Behavioral Risk Factor Surveillance System and Youth Risk Behavior Surveillance Survey

eTable 2. Cannabis Use Questions in Massachusetts Behavioral Risk Factor Surveillance System from 2011-2017