Discordant Reporting of Vaping of Cannabis among High School Seniors in the United States

Abstract

Introduction:

Reliable responses on surveys are important to ensure accurate estimates of drug use. This study compared prevalence of self-reported vaping of cannabis based on two separate questions on the same survey.

Methods:

We examined combined data from the 2017–2019 Monitoring the Future nationally representative survey of high school seniors in the US (N = 6,982). Students were first asked whether they had used cannabis in the past year via vaporizer in a section about cannabis use. Later on, students were asked about vaping behaviors, including cannabis vaping. Prevalence of self-reported vaping of cannabis was compared and a multivariable model delineated correlates of providing a discordant response – defined as reporting vaporizer use but not reporting vaping.

Results:

Estimated prevalence of use based on the question in the cannabis section was 11.9% (95% CI: 11.0–12.9) and prevalence based on the question in the vaping section was 13.2% (95% CI: 12.3–14.2). Among those reporting vaping of cannabis, 17.3% provided a discordant response. The corrected prevalence was 16.5% (95% CI: 15.5–17.6). Black students were more likely to provide a discordant report (aPR = 1.82, 95% CI: 1.23–2.69) and those reporting past-year vaping of nicotine (aPR = 0.27, 95% CI: 0.19–0.38) and/or nonmedical use of prescription stimulants (aPR = 0.44, 95% CI: 0.21–0.91) were less likely to provide a discordant report.

Conclusions:

Self-report of vaping of cannabis varies depending on whether it is asked in a section about vaping practices or about overall cannabis use (use via vaporizer). Survey researchers need to consider how to best query cannabis vaping practices on surveys.

Introduction

Accurate prevalence estimates are fundamentally important for sound epidemiological research on substance use and other health-related behaviors, particularly when such data are used to inform policy decisions. Self-report via surveys is the most common means of assessing the extent and nature of drug use. Advantages include being more practical, cost-efficient, and allowing the collation of more varied and detailed information when compared to alternative methods such as biological testing (1,2). However, care must be taken regarding the consistency of self-reported responses in order to ensure the collection of reliable and accurate data. For example, if a respondent initially does not report use of cannabis (veracity assumed), then that respondent should again respond non-affirmatively when subsequently queried about vaping of cannabis. Inconsistent responses to this end can reduce reliability, bias statistical estimates, and result in either under- or over-reporting of actual drug use (3-9). As such, findings from research that allow us to better understand the nature of inconsistent reporting can be leveraged toward improving the reliability of survey-based study designs.

The need for reliable surveys and accurate estimates may be further underscored for certain drugs, such as cannabis, owing to rapidly shifting policies in the United States (US) that favor legality and/or decriminalization of recreational use in as many as 16 states as of November 2020 (10,11). Not only has the prevalence of cannabis use and cannabis use disorders increased noticeably over the past decade (12-16), but the modes of cannabis consumption have also been shifting, with vaping of cannabis growing greatly in popularity among adolescents in recent years. More specifically, the most recent findings from Monitoring the Future (MTF), a nationally representative study of high school students in the US, found that the prevalence of past-year cannabis vaping spiked from 9.5% in 2017 to 22.1% in 2020 (17). In fact, among high school seniors (12^th graders), the abrupt increase in use between 2017 and 2020 includes the second largest single-year increase in use of any substance ever observed in the 46 years of MTF studies. MTF also estimated an increase in daily cannabis use among both 8^th, 10^th, and 12^th graders in 2019/2020, with vaping driving the increase in use by providing adolescents a convenient and clandestine means of using at home or in school settings (18).

While the current body of evidence lacks consensus as to whether vaping nicotine or cannabis-based products is definitively less dangerous than smoking combustible products (19), public health officials have expressed concern owing to data suggesting that cannabis vaping behaviors place people who use at greater risk for what has been dubbed electronic-cigarette or vaping product use-associated lung injury (EVALI). EVALI, a term coined by the Centers for Disease Control and Prevention (CDC), refers to vaping-related acute lung injuries ranging from mild dyspnea to acute hypoxemic respiratory failure requiring mechanical ventilation (20). In total, 2,807 cases of EVALI have been documented in the US as of February 2020, 68 of which have resulted in deaths (21,22). Of particular note is the fact that 82% of said cases were related to vaping products that contained tetrahydrocannabinol (THC) – the main psychoactive compound in cannabis (21). It appears that the current main causative factor may be Vitamin E acetate found in some THC cartridges (21). Individuals diagnosed with EVALI are more likely to exclusively vape products containing THC; they tend to vape more frequently at over five times per day and are also at greater risk of using bootleg or counterfeit THC-containing products from the black market which appear more likely to contain Vitamin E acetate (23-25). Aside from risk of EVALI, frequent vaping of cannabis also places people who use at increased risk for cannabis use disorder (26).

All things considered, there is a growing and urgent need to reliably collect accurate data on the vaping of cannabis. Vaping nomenclature varies as there are now many different types of devices (e.g., e-cigarettes, vape pens, vaporizers) with multiple generations and varying features, such as the ability to vape dried cannabis or fluid (and flavored or non-flavored) (27-29). For example, while many vaporizers and vape pens may appear to be similar devices, traditional cannabis vaporizers are used to vape dried cannabis, a feature only incorporated in recent generations of vape pens (27-29). As such, it is unknown whether adolescents report about use of such products consistently. In addition, not only is cannabis vaping a topical public health issue, but cannabis use in general is known to be commonly underreported, particularly when use was not recent (8,30-33). What has yet to be explored, however, is whether or not the nature of how survey questions are asked affect discordant responding of cannabis use – which can affect reliability of findings. MTF provides a unique opportunity to compare responses to separate questions on vaping cannabis on the same survey. Accordingly, the present study seeks to examine discordance between the prevalence of self-reported vaping as extracted from two separate sections in the same MTF survey – one section focusing on overall cannabis use and the other on vaping behaviors. This study also seeks to identify any potential correlates for discordant reporting of vaping cannabis.

Methods

Procedure

MTF is a nationally representative study of high school students in the US. A new cross-section is surveyed each year in approximately 130 public and private schools throughout 48 states using a multi-stage random sampling procedure (18). Six different survey forms are randomly distributed to seniors in randomly selected schools each year. This analysis focuses on Form 1, which includes two separate questions about whether students have vaped cannabis in the past year. We focused on combined data from 2017 to 2019 as questions about vaping were not added to MTF until 2017. Historically, MTF has administered surveys on paper, but in 2019, surveys were administered on electronic tablets. Data from the three most recent cohorts (2017–2019, N= 6,982) were aggregated for analysis. This secondary data analysis was exempt from review at the New York University Langone Medical Center Institutional Review Board.

Measures

Students were asked about past-year use of cannabis relatively early in the survey via the following question: “On how many occasions (if any) have you used marijuana during the last 12 months?” Response options were 1) 0 occasions, 2) 1–2 occasions, 3) 3–5 occasions, 4) 6–9 occasions, 5) 10–19 occasions, 6) 20–39 occasions, and 7) 40 or more. Those reporting use were asked, “What methods have you used for taking marijuana or hashish during the last year?” and students were prompted to check all responses that applied. “In a vaporizer” was the second response, alongside others including “smoking,” “eating in food,” “in a drink,” “dabbing a concentrate,” and “other.” Given that those reporting no past-year cannabis use were not asked about specific methods of use, we recoded a variable indicating such responses as reports of not vaping cannabis. Later in the survey, after being asked questions about past-year vaping of nicotine, students were asked, “On how many days (if any) have you vaped marijuana during the last 12 months?” “Response options were: 1) 0 days, 2) 1–2 days, 3) 3–5 days, 4) 6–9 days, 5) 10–19 days, and 6) 20 or more. Responses were recoded into a dichotomous variable indicating any use vs. no use (18). With respect to other drug use, students were also asked about past-year use of alcohol and cocaine, and about nonmedical use of prescription opioids and stimulants. Nonmedical use was defined as using without a doctor recommending use.

With respect to demographic characteristics, students were asked to indicate their age (pre-coded by MTF as <18 vs. ≥18 years), sex (i.e., male, female), and race/ethnicity (i.e., White, Black, Hispanic). Students were asked about the educational attainment of each parent and a variable was coded indicating the highest level of education attained by either parent. Students were also asked how many evenings per week they go out for fun or recreation. In addition, the population density were students resided was categorized by MTF as either a non-metropolitan statistical area (MSA), a small MSA, or a large MSA. Students were also coded as being surveyed via paper or electronic tablet.

Statistical analyses

We estimated past-year prevalence of vaping of cannabis based on three variables. First, we estimated use based on the item in the cannabis use section. Next, we estimated use based on the item in the vaping section, and then we estimated use based on a variable indicating self-report of vaping of cannabis on either item. Kappa interrater reliability between the items from the cannabis section and from the vaping section was calculated (34), although we utilized the vaping item from the cannabis section not including our added 0s for those not reporting cannabis use in order to prevent artificial inflation of agreement. Next, we created a variable indicating a discordant response between the two vaping items (limited to those reporting any vaping). Reporting vaping of cannabis on both items (or of vaping of cannabis on the second vaping item only) was coded as ‘no discord’. Reporting vaping of cannabis via vaporizer but not reporting overall vaping of cannabis later on the survey (excluding students who were missing data on one or both variables) was coded as ‘discord.’ In our main analysis, we did not code report of any cannabis vaping but no report of vaping via vaporizer as ‘discord’ based upon the logic that while all vaporizer use is vaping, not all vaping is via vaporizer. However, given that vaporizer and vaping terms tend to be used interchangeably (28), we also repeated our analyses with consideration for reported use via one vaping item but not the other as discordant report. Our coding of discordant report was based on previous research examining MTF data (5,6). Rao-Scott chi-square tests (35) were used to determine significant correlates of discordant report at the bivariable level. Generalized linear model using Poisson and log link was then used to delineate correlates of providing a discordant response (defined as reporting vaporizer use and then not reporting vaping) and this model produced adjusted prevalence ratios (aPRs) for each covariate. The model was then repeated as a sensitivity test considering reporting affirmatively on one vaping variable but not the other as discord.

Due to a high level of missingness of data, we imputed missing data for the multivariable model. However, a missing data indicator for race/ethnicity was included rather than imputing missing data for this variable because the majority of students with missing race/ethnicity data were not truly missing, but instead not available through MTF. Specifically, students not identifying as white, black, or Hispanic were coded as MTF as missing (36). We implemented multiple imputation via chained equations to handle missingness due to nonresponse. Predictors included all variables in the case-complete model. Ten datasets were imputed, on which we computed the multivariable model and combined results using Rubin’s Rules (37). All analyses were design-based for survey data (35) and utilized survey sample weights provided by MTF. Stata 13 SE software (StataCorp, College Station, TX) was used for analyses although R 4.0.2 software (R Core Team, Vienna, Austria) was used to compute Kappa using weighted data.

Results

Table 1 presents descriptive statistics for the full sample and for those reporting past-year vaping of cannabis on either item. Vaping of cannabis based on the question in the cannabis section (about use via vaporizer) was estimated to be 11.9% (95% CI: 11.0–12.9) and prevalence based on the question in the vaping section was estimated to be 13.2% (95% CI: 12.3–14.2). There was only moderate agreement (Kappa = 0.58, 95% CI: 0.53–0.62) between both items. Among those reporting having vaped cannabis via either item, 17.3% reported use via vaporizer but did not report overall vaping of cannabis. Among those reporting vaping of cannabis on either question, 42.8% reported vaping via one item but not the other. The corrected prevalence considering responses from both items was 16.5% (95% CI: 15.5–17.6). It should be noted that 3.5% (n= 30) of students (2.7% when considering the corrected estimate) who reported vaping cannabis later in the survey did not report having used cannabis in the past year earlier in the survey. Figure 1 demonstrates how all three estimates increased from 2017 to 2019 and all were significant linear increases across time (ps for trends <.001).

Table 1.

Characteristics of full sample and of students reporting past-year vaping of cannabis.

	Full Sample (n = 6,982)		Any Report of Past-Year Vaping of Cannabis (n = 1,123)
	Weighted % (95% CI)	N	Weighted % (95% CI)	N
Survey Type
Paper	82.8 (81.6–83.8)	5,901	77.0 (73.8–79.9)	895
Tablet	17.2 (16.2–18.4)	1,081	23.0 (20.1–26.2)	228
Age
<18	42.9 (41.4–44.3)	2,721	44.4 (40.8–48.1)	449
≥18	57.1 (55.7–58.6)	3,555	55.6 (51.9–59.2)	565
Sex
Male	46.7 (45.2–48.2)	2,849	54.3 (50.6–58.0)	528
Female	53.3 (51.8–54.8)	3,206	45.7 (42.0–49.4)	448
Race/Ethnicity
White	60.6 (59.0–62.2)	3,273	66.4 (62.6–70.1)	554
Black	14.4 (13.3–15.6)	711	9.1 (7.0–11.9)	70
Hispanic	24.9 (23.5–26.4)	1,220	24.4 (21.2–27.9)	198
Highest Parent Education
Less than High School	10.8 (9.8–11.8)	551	9.1 (7.2–11.4)	80
High School Diploma	18.5 (17.3–19.8)	997	16.7 (14.0–19.9)	141
Some College	18.5 (17.3–19.7)	1,046	19.1 (16.3–22.3)	176
College Degree	32.4 (31.0–33.9)	1,939	32.7 (29.3–36.3)	318
Graduate School	19.8 (18.6–21.0)	1,314	22.4 (19.6–25.5)	244
Evenings Out Per Week
0–1	41.0 (39.5–42.5)	2,354	29.4 (26.1–33.0)	257
2–3	46.8 (45.2–48.3)	2,731	52.7 (48.8–56.5)	484
4–7	12.3 (11.3–13.3)	714	17.9 (15.1–21.0)	170
Population Density
Non-MSA	19.3 (18.3–20.5)	1,216	15.8 (13.4–18.4)	159
Small MSA	45.8 (44.3–47.2)	2,970	47.1 (43.6–50.6)	474
Large MSA	34.9 (33.6–36.2)	2,796	37.1 (33.9–40.5)	490
Past-Year Drug Use
Vaping of Nicotine	26.5 (25.2–27.8)	1,675	74.2 (70.8–77.2)	763
Alcohol	58.4 (56.9–59.8)	3,888	94.6 (92.7–96.0)	1,008
Cocaine	2.0 (1.7–2.5)	116	9.6 (7.6–12.1)	84
Opioids (nonmedical)	2.8 (2.4–3.3)	183	9.2 (7.5–11.3)	100
Stimulants (nonmedical)	3.9 (3.4–4.4)	271	12.6 (10.5–15.1)	139
Cannabis (any)	35.4 (34.0–36.8)	2,310	97.3 (95.8–98.2)	1,046
Past-Year Vaping of Cannabis
Based on Cannabis Survey Section	11.9 (11.0–12.9)	817	76.6 (73.5–79.5)	817
Based on Vaping Survey Section	13.2 (12.3–14.2)	879	83.7 (80.8–86.3)	879
Based on Combined Items	16.5 (15.5–17.6)	1,123	–	–

Missing data were excluded and missingness was as follows: survey type (0.0%), age (10.1%), sex (13.3%), race/ethnicity (25.5%), parent education (16.3%), evenings out (16.9%), population density (0.0%), vaped nicotine (12.2%), alcohol (6.9%), cocaine (12.9%), opioids (11.3%), stimulants (8.5%), any cannabis (7.5%), vaped cannabis in cannabis section (8.7%), vaped cannabis from vaping section (12.4%), and vaping based on combined items (8.9%).

Figure 1.

Estimated prevalence of vaping of cannabis by year. Results demonstrate significant increases in estimated prevalence based on 1) the vaping item from the cannabis section (about vaporizer use), 2) the cannabis-vaping item from the vaping section, and 3) corrected prevalence based on a combination of both items (ps for all three trends<.001).

Table 2 presents results from bivariable tests and our multivariable model examining correlates of reporting use of a vaporizer but not reporting having vaped. Bivariable tests suggest that those reporting past-year vaping of nicotine (p< .001), past-year use of cocaine (p= .032), and/or nonmedical use of prescription opioids (p = .013) or stimulants (p < .001) were less likely to provide a discordant report. Bivariable tests also detected a difference based on race/ethnicity (p < .001). In the multivariable model, with all else being equal, compared to those identifying as White, those identifying as Black were more likely to provide a discordant report (aPR = 1.82, 95% CI: 1.23–2.69). In addition, those reporting past-year vaping of nicotine (aPR = 0.27, 95% CI: 0.19–0.38) and past-year nonmedical use of stimulants (aPR = 0.44, 95% CI: 0.21–0.91) were less likely to provide a discordant report.

Table 2.

Correlates of reporting cannabis use via vaporizer and then not reporting vaping of cannabis (among those answering both items).

	Full Subsample (n = 973) Weighted % (95% CI)	No Discord (n = 819) Weighted % (95% CI)	Discord (n = 154) Weighted % (95% CI)	aPR (95% CI)
Survey Type
Paper	75.7 (72.1–78.9)	73.4 (70.3–77.9)	82.3 (73.2–88.7)	1.00
Tablet	24.3 (21.1–27.9)	25.7 (22.1–29.7)	17.7 (11.3–26.8)	0.69 (0.45–1.06)a
Age
<18	43.5 (39.8–47.4)	43.7 (39.6–47.9)	42.7 (33.9–51.9)	1.00
≥18	56.5 (52.6–60.2)	56.3 (52.1–60.4)	57.3 (48.1–66.1)	1.16 (0.83–1.62)
Sex
Male	54.3 (50.4–58.1)	55.8 (51.5–60.0)	46.5 (37.3–56.0)	1.00
Female	45.7 (41.9–49.6)	44.2 (40.0–48.5)	53.5 (44.0–62.7)	1.31 (0.93–1.84)
Race/Ethnicity
White	68.3 (64.3–72.0)	71.0 (66.8–75.0)	54.6 (44.4–64.5)c	1.00
Black	8.3 (6.2–11.2)	6.4 (4.4–9.3)	17.7 (10.8–27.5)	1.82 (1.23–2.69)b
Hispanic	23.4 (20.1–27.1)	22.5 (19.0–26.5)	27.7 (19.5–37.9)	1.17 (0.73–1.85)
Highest Parent Education
Less than High School	9.0 (7.0–11.4)	8.9 (6.7–11.6)	9.5 (5.3–16.6)	1.00
High School Diploma	15.9 (13.2–19.1)	16.5 (13.4–20.0)	13.3 (8.0–21.2)	1.12 (0.56–2.24)
Some College	19.0 (16.1–22.4)	19.3 (16.0–23.0)	17.9 (11.8–26.2)	1.31 (0.67–2.54)
College Degree	33.0 (29.4–36.7)	33.0 (29.1–37.1)	32.8 (24.3–42.7)	1.36 (0.71–2.61)
Graduate School	23.1 (20.1–26.4)	22.5 (19.2–26.1)	26.4 (19.2–35.2)	1.39 (0.74–2.60)
Evenings Out Per Week
0–1	28.9 (25.4–32.7)	28.3 (24.5–32.3)	32.4 (23.6–42.7)	1.00
2–3	53.3 (49.3–57.3)	52.8 (48.4–57.2)	55.7 (45.9–65.2)	1.10 (0.75–1.61)
4–7	17.7 (14.9–21.0)	18.9 (15.7–22.6)	11.9 (7.2–19.0)	0.82 (0.45–1.50)
Population Density
Non-MSA	15.9 (13.4–18.8)	15.0 (12.4–18.1)	20.0 (13.8–28.1)	1.00
Small MSA	47.4 (43.7–51.2)	48.4 (44.3–52.5)	43.0 (34.3–52.2)	0.80 (0.53–1.22)
Large MSA	36.7 (33.2–40.2)	36.6 (32.9–40.4)	36.9 (28.3–46.5)	0.69 (0.44–1.09)
Past-Year Drug Use
Vaping of Nicotine	74.0 (70.6–77.2)	81.3 (78.0–84.2)	39.3 (30.8–48.4)c	0.27 (0.19–0.38)c
Alcohol	94.4 (92.3–96.0)	94.9 (92.6–96.6)	91.7 (85.8–95.4)	0.74 (0.48–1.16)
Cocaine	9.7 (7.6–12.3)	10.8 (8.3–13.9)	4.5 (2.0–9.9)a	0.85 (0.41–1.775)
Opioids (nonmedical)	8.7 (7.0–10.8)	9.8 (7.8–12.3)	3.3 (1.3–8.0)a	0.64 (0.28–1.46)
Stimulants (nonmedical)	12.2 (10.1–14.7)	13.9 (11.4–16.8)	3.7 (1.6–8.1)c	0.44 (0.21–0.91)a

In this analysis, discordant report was defined as reporting of vaping of cannabis via vaporizer and then not reporting vaping of cannabis later on the survey. Missing data were eliminated listwise in bivariable tests comparing discord vs. no discord. The multivariable model, however, utilized imputed data so no data were missing. MSA = metropolitan statistical area; aPR = adjusted prevalence ratio; CI: confidence interval.

^ap < .05

^bp < .01

^cp < .001

Supplemental Table 1 presents bivariable and multivariable models repeated, but with discord defined as reporting vaping via one variable but not the other. Multivariable results are comparable; however, race/ethnicity was not significant, and those reporting alcohol use and those surveyed via tablet instead of on paper were less likely to provide a discordant response.

Discussion

The popularity of vaping cannabis among adolescents has grown sharply in recent years, which has fueled public health concern due to the potential increased risk for developing EVALI (21,22). Further, frequent vaping of cannabis places people who use at increased risk for cannabis use disorder (26). Surveys are a practical means of estimating the prevalence of vaping of cannabis, they may be susceptible to discordant responding, which reduces the reliability of findings and leads to under- or over-reporting of prevalence estimates. The MTF annual study provides a unique opportunity to compare responses to queries regarding vaping of cannabis from two separate sections on the same survey (a cannabis use section versus a vaping behavior section) and identify potential discordant reporting.

Overall, our study found extensive discordant reporting of vaping of cannabis, with 17.3% of those reporting having vaped cannabis using a vaporizer providing a contradictory report. However, this was a conservative estimate based on the assumption that students interpreted the first question about vaporizer use as not specifically applying to vape pens. About four out of ten (42.8%) of those answering both vaping questions provided a contradictory response as per the more liberal definition of discord. Such discord detected during our analyses led to only moderate test-retest reliability of findings. This is of concern since the prevalence figures presented in the annual MTF reports are only based upon the question from the vaping behaviors section of the survey. When considering responses from both sections, and assuming that estimates are based on underreporting more so than overreporting, we estimate that past-year prevalence of vaping cannabis is approximately 3% higher than originally estimated (ranging from 2.5% to 4.0% in individual years). This would suggest that vaping of cannabis among adolescents is actually more common than originally estimated. The assumption that vaping of cannabis is underreported in reports focusing on the full sample is based on previous findings demonstrating that cannabis use is commonly underreported on surveys (8,30-33). This is coupled with the fact that some students in the present study reported vaping of cannabis but did not report any past-year cannabis use earlier in the survey. In the grander scheme, this general finding of discordant reporting points to an area that warrants additional consideration from survey researchers tasked with deciding how to best query cannabis vaping.

To inform such efforts in terms of possible reasons underlying discordant reporting, we further sought to identify potential correlates of discordant reporting. Our findings suggest that students who also reported vaping nicotine were less likely to provide a discordant response as it pertains to vaping cannabis. The same can also be said among students who also reported nonmedical use of prescription drugs such as opioids and stimulants. This corroborates results from previous studies that have found that respondents who have more experience with other drugs are less likely to underreport use of drugs (30), including among national samples of high school students who reported using opioids or amphetamine nonmedically (5,6). One previously posited explanation is that people who use other drugs tend to perceive less stigma surrounding drug use and may be less susceptible to reporting bias as a result (5,38,39). A second and potentially complementary explanation is that more experienced drug users may be more knowledgeable about drugs and hence more likely to answer correctly on drug surveys and are less likely to respond discordantly due to information bias (5,40,41).

Likewise, our findings suggest that Black students were more likely to provide a discordant response related to vaping cannabis compared to White students. Previous studies investigating MTF data have also demonstrated higher prevalence of discordance among Black students reporting nonmedical use of opioids and amphetamine (5,6). However, in a recent study examining self-correction of self-reported marijuana use on a survey, race/ethnicity was not found to be related to inaccurate reporting of marijuana use (30). At this time, it is unknown why Black students may be at greater risk for providing a discordant response on national surveys.

It is worth noting that as of 2019, MTF surveys have been administered using electronic tablets. Our findings also suggest that that those surveyed on tablets were less likely to provide a discordant response compared to those surveyed on paper in the samples prior to 2019. This finding does not corroborate findings from a recent study conducted by MTF investigators, which found that there were no significant differences in prevalence of past-year drug use between paper and electronic survey modes (42). While the reason for this difference remains unknown, it is likely that surveys will continue to be administered electronically for the foreseeable future, which has important implications for how questions are sequenced. For example, electronic surveys administered through tablets permit the use of skip-logic. In skip-logic methodology, an individual’s response to a question determines whether relevant follow-up questions will be asked. Aside from reducing participant burden stemming from being asked irrelevant questions, this method is advantageous in allowing researchers to conveniently query a wide variety of phenomena (43), and making data analysis more streamlined since skip-logic prevents discordant responses (e.g., reporting no lifetime cannabis use, but then later reporting past-year cannabis use). The latter point may also be disadvantageous insofar it masks discordance that may otherwise have been informative. In addition, erroneously not reporting cannabis use in the cannabis section did not prompt students to ask whether they vaped cannabis, so relying on a follow-up question to query vaping may have further contributed to underreporting. MTF appears to use skip-logic for some items and it allows students to skip questions (hence the continuation of missing data in 2019), but it can be difficult to use skip-logic or piping methods across sections with only minor overlap. We recommend providing the option to type in drugs used at the end of a survey as a means of detecting unintentional non-affirmative responses from earlier in the survey.

Ultimately, our results suggest that asking about vaping of cannabis along with questions about general vaping practices was associated with higher prevalence of self-reported use compared to a question asked in a section focusing on cannabis use practices. We do not believe results are based on an order effect because reported prevalence was higher later in the survey rather than earlier in the survey. While more research is needed to determine why there was discord, it is possible that some students who solely vape cannabis may inadequately complete questions in the cannabis use section as they may not consider themselves traditional “cannabis users.” To some extent, this may also be related to known psychological phenomena of priming and framing in survey designs. For example, questions in the section specifically on vaping behaviors are framed in terms of vaping practices as opposed to overall substance use, which may prime students to think about vaping more intimately than a vaping query that appears in the context of general drug use (44,45). Further, use of the term “vaporizer” could have led to some confusion among students. While many vaporizers and vape pens often look similar, the majority of older vape pens cannot vaporize dried cannabis like traditional vaporizers can (27-29).

Future studies planned with a specific aim to deduce predictors of discordant report are needed – particularly those that utilize theoretical frameworks such as the impression management model (46). Impression management is a conscious or unconscious process by which individuals attempt to influence perceptions of themselves by controlling the information that they provide (47). This theory posits that situational perceptions and willingness to report honestly in particular are determinants of intention to report honestly (which, in turn, predicts honest reporting) (46,48,49). Furthermore, we believe such a model should include variables assessing common biases including comprehension, memory difficulty, disabilities that may affect survey-taking (e.g., visual, hearing, understanding/recall), trust toward researchers, and social desirability, which often predict underreported drug use (when comparing survey responses and biological test results)(50).

Limitations

Only Form 1 (out of six survey forms) contains both vaping questions so it was only possible to examine data for a sixth of the annual samples. We based our analyses on the assumption that discordant responses represented underreporting on one of two vaping items, although it is possible that discordant responses (or even concordant responses) are a result of some students overreporting. While adults have been found to be less likely to overreport or to provide mischievous survey responses, this is more likely to occur among adolescents (51-53). MTF has a substantial amount of missing data, even in the most recent cohort which utilized tablets instead of paper surveys. Although we used imputed data for the multivariable model, we had to rely on case-complete data for univariable and bivariable analyses and for calculations of discordant reporting. Our discord variable did not include those who provided data on one vaping variable and did not answer the other. Finally, students who dropped out or who were chronically absent were not surveyed and this can impact estimates of drug use.

Conclusion

When querying vaping of cannabis, it appears that asking about vaping of cannabis along with questions about general vaping practices elicits responses yielding higher prevalence than vaping follow-up questions stemming from self-report of cannabis vaping practices. More research is needed to further investigate how different forms of survey items elicit different responses regarding drug use, but we believe national estimates of vaping of cannabis may in fact be underestimates.