“Buzzwords”: Crowd-Sourcing and Quantifying U.S. Young Adult Terminology for Subjective Effects of Alcohol and Marijuana Use

Abstract

Prevalence of heavy alcohol use remain high and daily marijuana use is at an all-time high in young adults. As perceptions of drug effects may guide risky decision-making, understanding subjective feelings for alcohol and marijuana use is critical. Existing laboratory-based and diary metrics (0 to 100 rating of, “how drunk/high do you feel?”) may be problematic in differentiating levels of subjective effects. Measures incorporating contemporary language may better capture subjective feelings in experimental and ambulatory assessment designs. We developed two sliding scales based on crowd-sourced and rank-ordered feelings of subjective alcohol and marijuana effects. Two U.S. young adult (18–25 years) samples were drawn from Amazon’s Mechanical Turk (MTurk). In the first study, 323 (53.6% women, 68.4% White, M age=23.0 years) alcohol and marijuana users provided words to describe subjective effects from alcohol use and from marijuana use. In a second study, 289 (46.4% female, 66.4% White, M age=23.0 years) users rank-ordered the most common terms to correspond with subjective levels. The sliding scale for alcohol effects resulted in four anchors ranging from 0 to 100: slightly buzzed, tipsy/“happy”, drunk, and wasted. The sliding scale for marijuana effects also had four anchors: relaxed, calm/chill, high, and stoned/baked. By incorporating broader and more contemporary subjective effects language, our metrics may better capture variability in young adults’ perceived subjective effects from alcohol and marijuana use. Future work could build upon these findings by pairing sliding scales with measures of actual intoxication to understand within- and between-person covariation in subjective effects, actual impairment, and consequences.

The prevalence and patterning of young adult substance use is rapidly changing in the U.S. Prevalence of daily marijuana use (use on 20+ occasions in the prior 30 days) are at an all-time high and perceptions of risk attributed to regular marijuana use are at an all-time low in this age group (Schulenberg, Johnston, O’Malley, Bachman, Miech, & Patrick, 2018). Frequent shifts in the legalization of recreational and medicinal marijuana use, the invention of nuanced delivery systems for marijuana use, and historically higher THC concentrations in cannabis preparations (Chandra et al., 2019) warrant further understanding of young adult marijuana use. Heavy drinking among young adults remains a critical public health concern with approximately 40% of 21/22 year-olds reporting binge drinking in the last two weeks (Schulenberg et al., 2018).

There are many negative outcomes associated with heavy and frequent alcohol and marijuana use – separate or combined – including academic/occupational problems, self-care and self-perception concerns, unsafe driving, and risky sexual behavior (Subbaraman & Kerr, 2015; White & Hingson, 2013). Much research has focused on identifying “binge” thresholds for safer drinking (National Institute on Alcohol Abuse and Alcoholism, 2004) and on quantifying marijuana use (Prince, Conner, & Pearson, 2018). There are, however, challenges with classifying and quantifying risky substance use (e.g., Linden-Carmichael, Russell, & Lanza, 2019; Pearson, Kirouac, & Witkiewitz, 2016; Prince et al., 2018). For example, although cut-points for binge drinking were designed to be approximate measures of the number of drinks one can consume before reaching the legal driving limit, binge drinking does not consistently correspond with achieving a blood alcohol concentration of 0.08% (see Pearson et al., 2016). Related to marijuana, there are inconsistencies with respect to level of high achieved with differential product use (e.g., edibles, vaping, smoking; Cooper & Haney, 2009; Spindle et al., 2018). Moreover, self-report data are fraught with a number of recall- and accuracy-related issues.

Although measurement of objective levels of intoxication are critical to understanding patterns of use such as speed or peak level of intoxication in the absence of recall biases, there are several problems with solely relying on objective measures. Oftentimes, decisions to engage in risky behavior such as deciding whether one is sober enough to drive are not based on how intoxicated they actually are but rather how intoxicated they feel (Quinn & Fromme, 2012). As one example, mixing an energy drink with alcohol can counteract the sedative effects of alcohol without reducing actual intoxication (Marczinski & Fillmore, 2006). Field studies show that individuals who consumed these beverages were at a four-fold increase of intending to drive home after drinking than individuals who consumed other types of alcohol (Thombs et al., 2011). The potential disconnect between actual and subjective intoxication suggests that it may be equally, if not more critical to assess perceived effects when assessing and intervening on real-time behavior.

Common assessments focused on identifying subjective feelings include the Biphasic Alcohol Effects Scale (BAES; Martin, Earleywine, Musty, Perrine, & Swift, 1993), Subjective Effects of Alcohol Scale (SEAS; Morean, Corbin, & Treat, 2013), and Subjective High Assessment Scale (SHAS; Judd, Hubbard, Janowsky, Huey, & Atewall, 1977; Schuckit & Gold, 1988). These scales measure differential perceived effects including stimulation, sedative effects, and affective states (high/low arousal positive/negative). In the BAES, for example, participants separately indicate the extent to which they feel each adjective (e.g., “inactive”, “elated”, “talkative”) on a scale ranging from 0 (not at all) to 10 (extremely). The BAES and SEAS each contain 14 items, which could be burdensome to complete in surveys with time constraints, such as day- or moment-level surveys designed to intervene on behavior in real time. Shorter, one-item sliding scales have been developed for assessing a related construct of subjective intoxication by asking, “How drunk (high) do you feel?” or “How intoxicated do you feel?” with anchors ranging from 0 (completely sober) to 100 (extremely drunk/high) (e.g., Bravo, Pearson, Conner, & Parnes, 2017; Quinn & Fromme, 2011). While these measures may assess extreme high and extreme low levels, some evidence suggests that metrics of how “drunk” or high someone felt does not distinguish moderate from high levels of perceived effects (Levitt, Sher, & Bartholow, 2009). As a result, participants may be assigning arbitrary values or treating such metrics as dichotomous indicators of whether they were drunk/high, rather than the varying degree of perceived effects. Levitt et al. (2009) and Levitt, Schlauch, Bartholow, and Sher (2013) demonstrated that there are a variety of terms young adults use to describe subjective states, and that young adults use different words to describe varying degrees of subjective effects. Use of familiar terms at various subjective states lends itself to the creation of sliding scales with multiple anchor points using contemporary language. Such a scale can be easily implemented in laboratory-based or diary studies with the goal of capturing subjective effects across a continuum.

The current study had three goals. First, in Study 1, we used Amazon’s Mechanical Turk (MTurk), a crowd-sourcing marketplace, to ask participants to generate words capturing perceived effects when using alcohol and, separately, marijuana, at the participants’ perception of a low, moderate, and heavy level. Second, in Study 2, we provided the most commonly used terms for alcohol and for marijuana to a second MTurk sample and asked them to rank-order words they would use based on level of use (low levels to high levels) from using both substances. Third, we developed two sliding scales with multiple meaningful anchor points to assess subjective effects of alcohol and marijuana use.

Study 1: Word Generation

Method and Results

Participants and procedure.

Participants were recruited through MTurk, an online crowdsourcing marketplace in which “requesters” (researchers) can pay “workers” (participants) to complete online tasks, such as survey questionnaires. After workers complete a task, requesters evaluate their performance (e.g., amount of missing data, whether answered “attention check” questions correctly) and approve or reject their work. Only participants whose work is approved receive payment for their participation. MTurk is commonly used for numerous questionnaire and experimental studies (see Chandler & Shapiro, 2016) including several focused on addictive behaviors (e.g., Amlung et al., 2019).

To be eligible, participants must (1) have been between the ages of 18–25 years, (2) have reported past-month binge drinking (4+/5+ drinks in one sitting for women/men), (3) have reported past-month simultaneous use of alcohol and marijuana, (4) live in the U.S., (5) speak fluent English, (6) have a task approval rating of >95%, (7) have completed >50 tasks on MTurk previously. Study 1 (n = 323) participants were most likely to be female (53.6%), White (68.4%), graduates of a 4-year university (41.2%), and from the South (37.2%). M age was 23.0 (SD = 1.8). Participants reported a weekly average of 14.66 (SD = 9.83) drinks and 9.65 (SD = 10.21) joints.¹

After providing informed consent, eligible participants completed several online questionnaires that took approximately 10 min. to complete. Based on prior suggestions (Abbey & Meloy, 2017), attention checks were placed throughout the survey; work was approved if they answered >2 of the 3 attention checks correctly and if they provided plausible responses. Participants received $1.00. The university’s institutional review board approved the current study. All APA ethical guidelines were followed. Data were collected from July-August 2018.

Measures.

Participants provided words they would use to describe subjective feelings at different levels of alcohol and, separately, marijuana use. Participants wrote one- or two-word phrases to describe their level of impairment when using each substance without using other substances. Participants were instructed to provide at least two (max of four) feelings each for lighter, moderate, and heavier occasions of alcohol alone and marijuana alone (e.g., “When I’m drinking a smaller amount of alcohol [maybe one or two drinks], I would describe my level of impairment as feeling…”).

Participants completed the Daily Drinking Questionnaire (DDQ; Collins, Parks, & Marlatt, 1985), a calendar grid assessing typical weekly alcohol use. A modified DDQ assessed the number of joints typically used during a week. Participants completed a brief demographic questionnaire regarding their sex, age, race/ethnicity, college education status, and region.

Data analysis and results.

Word data generated by participants of Study 1 were cleaned to maintain consistency across participants (e.g., “buzzed”, “buzz”, and “buzzing” were converted to “buzzed”). Words were ordered in terms of frequency used; the top 20 words were identified separately for alcohol and for marijuana. The 10 most frequently used words were automatically included in the list of words we provided to Study 2 participants for rank-ordering. As the majority of generated words reflected lighter use episodes, to provide potential upper limits for the scales, we included three additional words for each substance that were less frequently used but described heavier episodes (e.g., “hammered” for alcohol; “paranoid” for marijuana) in the list given to Study 2 participants. As a result, Study 2 participants were provided with 13 words describing alcohol-related effects and 13 words describing marijuana-related effects. The frequency of which Study 1 participants generated these words is provided in Table 1, in order of frequency.

Table 1.

Self-Generated (Study 1) and Self-Ranked (Study 2) Subjective Effect Terms for Alcohol Use and for Marijuana Use

Study 1: Impairment Terms Generated for Alcohol Use and Marijuana Use (n = 323)
Alcohol Terms		Marijuana Terms
Word	Number (Percentage) of People Who Generated Word	Word	Number (Percentage) of People Who Generated Word
Buzzed	188 (58.2%)	High	158 (48.9%)
Drunk	114 (35.3%)	Relaxed	113 (35.0%)
Tipsy	102 (31.6%)	Sleepy/Tired	88 (27.2%)
Happy	98 (30.3%)	Calm/Chill	87 (26.9%)
Relaxed	78 (24.1%)	Stoned	87 (26.9%)
Nice/Good/Fine	66 (20.4%)	Happy	86 (26.6%)
Dizzy	58 (18.0%)	Buzzed	74 (22.9%)
Sleepy/Tired	56 (17.3%)	Nice/Good/Fine	64 (19.8%)
Calm/Chill	52 (16.1%)	Sober/Nothing	63 (19.5%)
Wasted	44 (13.6%)	Hungry	45 (13.9%)
Loose	33 (10.2%)	Baked	29 (9.0%)
Hammered	28 (8.7%)	Dizzy	19 (5.9%)
Blackout/Passed out	24 (7.4%)	Paranoid	19 (5.9%)
Study 2: Rank-Ordered Impairment Terms for Alcohol Use and Marijuana Use (n = 289)
Alcohol Terms		Marijuana Terms
Rank-Ordered Word (Effective N)	M Placement in Level of Impairment Due to Alcohol Use	Rank-Ordered Word (Effective N)	M Placement in Level of Impairment Due to Marijuana Use
Buzzed (242)	36.34	Sober/Nothing (109)	26.23
Nice/Good/Fine (238)	38.46	Relaxed (269)	37.49
Tipsy (237)	41.18	Calm/Chill (256)	43.59
Relaxed (229)	41.60	Nice/Good/Fine (240)	43.61
Calm/Chill (206)	42.34	Buzzed (139)	44.76
Happy (244)	43.27	High (264)	45.29
Loose (140)	48.67	Happy (237)	47.62
Drunk (255)	51.60	Hungry (216)	58.10
Dizzy (173)	60.31	Stoned (229)	59.68
Sleepy/Tired (182)	60.77	Dizzy (90)	60.36
Hammered (165)	63.98	Baked (166)	60.78
Wasted (188)	65.29	Sleepy/Tired (197)	64.71
Blackout/Passed out (148)	77.95	Paranoid (121)	67.17

Note. In Study 2, effective N refers to the number of participants (out of 289) who used each word in their own rank-ordered list. Words were separately selected as anchor points for the alcohol effects scale and the marijuana effects scale. The most commonly used words among lower, moderate, high, and very high levels were selected as anchor points. Words that did not reflect meaningful feelings of impairment or behavioral results of use were excluded.

Study 2: Rank-Ordering Words by Level of Subjective Effects

Method and Results

Participants and procedure.

Study 2 (n = 289) participants were recruited through MTurk using the same eligibility criteria, attention checks, and compensation. Participants were most likely to be male (53.6%), White (66.4%), graduates of a 4-year university (46.7%), and from the South (41.2%). M age was 23.0 (SD = 1.9). Participants reported a weekly average of 16.85 (SD = 12.15) drinks and 11.05 (SD = 12.12) joints. Data were collected in August-September 2018.

Measures.

Study 2 participants rank-ordered the 13 commonly used words generated by Study 1 in order of impairment. Participants were provided with a list of commonly used words and were asked to drag and drop each word they would use to describe their feelings of impairment, ranging from lower to higher levels. Participants were also able to place words into a designated box for words they personally would never use.

Data analysis and results.

To account for inconsistencies in the total number of words individuals in Study 2 selected to rank-order, we transformed participant data to create a consistent metric across participants. We (1) determined the total number of words each person used from the 13 words for alcohol and, separately, 13 words for marijuana; (2) calculated person-specific percentiles for each word based on its rank-order and the number of words used by each person; and (3) calculated the mean percentile, minimum, maximum, and SD for each word across participants. Mean percentiles are listed in Table 1.

Anchor Point Selection for Sliding Scales

Method and Results

After including the most common words on the same metric, we simplified each scale to 4 anchors for usability purposes in daily or ecological momentary assessment (EMA) surveys. We set the lowest (0) scale anchors at the most commonly used word with the lowest M percentile. Words that did not reflect meaningful feelings of impairment (“nothing/sober”) or behavioral results of use (“blackout/passed out”, “sleepy/tired”) were excluded. For marijuana, “stoned” and “baked” were combined as they were common and close in M placement (59.7 and 60.8, respectively). The most commonly used words from the second and third quadrants were included as individual anchors; words that were close in M placement and relatively common were combined (“tipsy” and “happy”). This resulted in a sliding scale with four evenly-spaced anchor points for alcohol-related impairment ranging from slightly buzzed (0), tipsy/“happy”, drunk, and wasted (100) and a separate sliding scale with four evenly-spaced anchor points for marijuana-related impairment ranging from relaxed (0), calm/chill, high, and stoned/baked (100). A ruler-based 0 to 100 metric was selected given its improved validity and reliability over Likert-scale measures (Yusoff & Janor, 2014), ease in administration (Yusoff & Janor, 2012), and increased precision (i.e., more response options). The final sliding scales for subjective alcohol and subjective marijuana impairment are shown in Figures 1a and 1b, respectively.

Figure 1a and 1b.

Four-anchored sliding scales for assessing subjective feelings (a) when using alcohol and (b) when using marijuana.

Discussion

Our study captured contemporary language of subjective effects related to alcohol and marijuana use, culminating in two sliding scales ranging from low to high levels of subjective effects for both substances. This study builds upon prior work understanding the language of alcohol effects more broadly (Midanik, 2003; Ray, MacKillop, Leventhal, & Hutchison, 2009; Reich, Ariel, Darkes, & Goldman, 2012), assessment of subjective effects (e.g., Judd et al., 1977; Martin et al., 1993; Morean et al., 2013; Schuckit & Gold, 1988), and also work that assessed the commonality of various terminology of alcohol’s effects (Levitt et al., 2009). Although adding anchors to a simple 1-item sliding scale may appear to be only an incremental improvement, methods developed by Thurstone in the 1920s of gathering various opinions toward a phenomenon and submitting a refined list of terminology to “judges” for feedback has long been thought to be an important, albeit laborious, process to scale development (see McIver & Carmines, 1981). Consequently, our study contributed to existing literature in several ways. First, whereas Levitt et al. identified terms through the use of dictionaries and laboratory discussions, we asked young adult alcohol and marijuana users to self-describe subjective feelings. This offers a potentially improved approach to measuring young adults’ subjective effects. Also, we surveyed contemporary language; words in prior work were generated >10 years ago. Second, we asked young adults to rank-order these words to understand how words are used at different stages of drinking or using marijuana. Third, far less research has focused on self-described terminology for marijuana-related impairment. As marijuana rates continue to increase among young adults especially (Schulenberg et al., 2018), understanding how individuals describe how they feel when using marijuana is critical for understanding and assessing subjective perceptions.

It is worth noting that in examining participant word generation data, few participants self-generated many of the terms used in common assessments for subjective effects. Items in the SEAS (Morean et al., 2013) include words such as “woozy”, “wobbly”, “demanding”, and “talkative”; the BAES (Martin et al., 1993) includes “heavy head”, “sluggish”, and “stimulated”. Levitt et al. (2009, 2013) demonstrate the importance of including contemporary language in metrics assessing alcohol-related constructs. By using terms for subjective effects that young adults do not commonly use, young adults may have difficulty self-expressing their feelings on these metrics. Moreover, although applied specifically to subjective intoxication, it is noteworthy that the commonly used words “drunk” and “high” were not the top level of our scales. A common assessment method for subjective intoxication is to ask participants how “drunk” or “high” they feel after using substances. Given findings from Levitt et al. (2009), these metrics may fail to differentiate moderate and high levels of subjective effects. In light of the words selected and rank-ordered by our sample, it is possible that current metrics are not fully capturing the continuum of heavy substance use with anchors of “drunk”.

The ultimate goal of this project is for researchers to implement these scales in a daily or fine-grained EMA study design. A critical first step is to test whether this scale effectively captures a range of drinking. By providing familiar terms as anchors of subjective effects on a continuum, we hope to better understand subjective states. Another step for future work is to identify how subjective feelings align with actual intoxication. From a public health perspective, this information could be critical in several ways. Many decisions could be made by young adults based on how they feel after drinking, rather than their actual intoxication. Understanding how young adults are feeling and what words they would use to describe their subjective state at various levels of use – and how this maps onto driving or willingness to drive while impaired – could inform prevention and intervention efforts for high-risk individuals. Through a diary study, this can be achieved by asking participants how many drinks they consumed last night to obtain an estimated blood alcohol concentration (eBAC) and examining what words individuals use at different levels of eBAC. With an EMA study design, this could be achieved by participants wearing an alcohol sensor while reporting on their moment-level subjective states. This design would allow for an examination of the dynamic covariation between actual and subjective effects throughout a drinking episode, and a personalized understanding of words that one uses when at a particular level of impairment. Such designs also would allow for validation of our scale development in terms of the rank ordering of words.

There are a few limitations that should be noted. First, the parent study was focused on simultaneous alcohol and marijuana (SAM) users; thus eligibility criteria included past-month SAM use. Most who use alcohol and marijuana engage in SAM use (Subbaraman & Kerr, 2015), however findings may not generalize to the relatively fewer individuals who engage in concurrent, but not simultaneous use. Relatedly, given this inclusion criterion, it is possible that some participants may have had limited/no experiences with using only marijuana. Second, our sample consisted of Amazon MTurk workers. While this sample has strengths in representation of multiple states in the U.S. and education statuses, MTurk samples have limitations. MTurkers have higher negative affect and lower social engagement (McCredie & Morey, 2018) than the average population. Thus, replication of these findings with national samples may be beneficial. Relatedly, findings from our study may not generalize to samples outside the U.S. or to other age groups. The goal of this study was to develop a scale that was useful for the general population of U.S. young adults; thus, the contemporary language used in this sample may not be applicable to other populations. A similar study could be conducted to develop scales that capitalize on language used in other populations. Post-hoc analyses do, however, reveal that each word did not differ by U.S. region with the exception of “tipsy.” Third, substance use data relied on self-report. Such self-reported data may be subject to recall bias and social desirability concerns. Fourth, our measures describing the typical amount of marijuana used focus primarily on number of joints used. Although participants were asked to consider the number of joints they could have rolled if they had used a pipe/blunt, there are many options for marijuana delivery systems that were not reflected in this measure. Future diary work using this scale to predict level of use should consider number of hits and the variety of routes of administration.

Public Health Significance:

Using contemporary young adult language, we developed two sliding scales to assess subjective feelings for alcohol and marijuana use. These metrics may better capture variability in young adults’ subjective feelings of alcohol- and marijuana-related effects.