Parameters of EMA Compliance and Self-Reported Reactivity in a Longitudinal Study of Young Adult Cannabis and Tobacco Co-Users

Abstract

Background:

Co-use of cannabis and tobacco has become increasingly popular among young adults. Interactive voice response (IVR) based ecological momentary assessment (EMA) allows for measurement of behavior in or near real-time, but has limitations including non-compliance, missing data, and potential for reactivity (e.g., behavior change) from frequent assessments.

Methods:

This study examined tobacco and cannabis use characteristics and factors associated with IVR compliance and self-reported reactivity in 97 young adults who reported cannabis and tobacco co-use at baseline and completed daily IVR surveys of co-use behavior at three random times per day for 28 days.

Results:

Overall IVR compliance was 55%, with a modal compliance of 60%. Compliance rates did not differ across morning, midday, and evening surveys, but significantly declined over time. The sample was divided into high frequency responders (>70% calls completed, n=35) and low frequency responders (<70%, calls completed n=62). There were no differences between high and low frequency responders on any baseline demographic, tobacco use (nicotine dependence severity), alcohol, or cannabis use characteristics (past 30-day frequency of use). Participants were receptive to IVR-based EMA monitoring and, 16.5% reported purposely decreasing nicotine/tobacco use due to monitoring, while 19.6% reported purposely decreasing cannabis use, which predicted lower cannabis use post-EMA monitoring.

Conclusions:

Real-time assessment of co-use behavior among young adults does not appear to be impacted by specific demographics or substance use severity (nicotine dependence, heavy drinking). Data suggest some predictive utility of IVR-based EMA monitoring on short-term behavior change. More intensive approaches are needed to improve compliance among young adult cannabis and tobacco co-users.

Over the last 15 years, legalization and changing positive attitudes towards cannabis use have been accompanied by increased prevalence of adult use in the US. According to the 2019 National Survey on Drug Use and Health, adult past year cannabis use has increased from 10.4% in 2002 to 18.0% in 2019. Past-month cannabis use is also two times higher for young adults (youg adults; ages 18–25) than the national average for all adults (23.0% vs 11.9%, respectively) and has increased in this age group over the last 15 years (National Academies of Sciences & Medicine, 2017). There is considerable overlap of cannabis use with tobacco products, particularly in this age group (Cohn et al., 2016; Cohn, Johnson, et al., 2018). Rates of co-use have also increased over time in young adults (Schauer et al., 2015a, 2015b), with approximately 20% of young adults reporting past month co-use (Cohn, Abudayyeh, et al., 2019). Tobacco and cannabis co-use is correlated with a variety of mental and physical health consequences including cannbis and tobacco dependence, cancer, and psychiatric symptoms (Botchway & Deshpande, 2015; Cohn et al., 2021; Gage et al., 2015).

One promising strategy for precisely measuring cannabis and tobacco co-use is the use of ecological momentary assessment (EMA), which allows for the collection of cognitive, affective, and behavioral phenomena in natural settings, and in, or near real-time. As such, processes can be captured “in the moment” and closer to real-time occurrences. Because data are “time-stamped”, behaviors can be recorded and verified as occurring at a particular point in time, relative to others, reducing recall bias and maximizing causal inferences (Conner et al., 2009; Tennen & Affleck, 2002). Interactive Voice Response (IVR) is one EMA method that uses automated pre-recorded survey items to which participants respond by pushing buttons on the keypad of their phone (Ankawi et al., 2022; Cheong & Tucker, 2022; Gorfinkel et al., 2021). Lastly, because behavior is collected over multiple days, EMA approaches increase the number of data points one has to predict an outcome (Barta & Tennen, 2008; Nelson & Hayes, 1979). This also allows for detailed consideration of the variability in substance use patterns that occurs within-individuals and in a longitudinal fashion which eliminates sources of confounding when data are aggregated. This is notable, given discrepancies between real-time reporting surrounding substance use behavior and recall-based reporting of the same events (Carney et al., 1998; Shiffman et al., 1997; Todd et al., 2005). Recall methods are less accurate than prospective daily assessments because they are prone to cognitive heuristics and emotional experiences that bias information retrieval and processing (Piasecki et al., 2007). Because of these methodological strengths, EMA data collection can add significant clinical and predictive value to our understanding of processes related to substance use behavior (Piasecki et al., 2007).

EMA does have limitations worth noting, including missing data, the potential for low compliance, and possible reactivity or behavior change in response to frequent and repeated assessments of the same behavior. To ensure valid statistical analysis and representativeness of one’s data, it is important to understanding sources linked to the occurrence of missing data and possible reactivity, so that these factors can be minimized in the study design and controlled for in analytical models. One study of smokers who engaged in risky drinking found that 80% of respondents indicated increased awareness of their behavior and 40% reported some form of behavior change specifically in response to completing twice daily IVR assessments for 28 days (Cohn, Elmasry, et al., 2018). Similarly, another study of heavy-episodic drinkers showed that mobile assessments both with and without intervention were associated with decreased heavy-episodic drinking when compared to minimal assessments with and without intervention (Witkiewitz et al., 2014). Other studies have reported either no significant reactivity to daily monitoring, or that when reactivity has been found to exist (M. R. Hufford et al., 2002), it accounts for only a small proportion of the variance in behavior change (Clifford et al., 2007; Maisto et al., 2007; Rowan et al., 2007). According to Barta and colleagues, reactivity to EMA survey can be mitigiated when multiple behaviors are monitored (Barta et al., 2012; Hufford et al., 2002), as respondents are likely unaware of the specific behaviors they are being “primed” to monitor.

Statistical validity may also be impacted by data that are missing systematically or at random. Data that are missing at random can result in decreased statistical power (Graham, 2009); however, low compliance may be indicative of systematic bias challenging the representativeness of the sample (Stone & Shiffman, 2002). Efforts to minimize bias and improve statistical power prompt examination of factors theorized to be associated with missing data and low compliance. Frequent assessment schedules may place a greater burden on participants and thus reduce overall compliance (Robbins & Kubiak, 2014), while too few may result in participants forgetting to notice prompts. Daily interruptions over long periods caused by random prompts can also be inconvenient and contribute to decreased compliance over time (Burke et al., 2017; Johnson et al., 2009). In some studies with multiple weeks of EMA, lower compliance has been reported for later weeks (Hoeppner et al., 2014; Yang et al., 2015). Additionally, IVR technology instructs participants to push buttons on the keypad of their phone to answer pre-recorded survey questions. This requires continuous audio attention and is less conducive to multi-tasking when compared with other more visual EMA methods such as SMS (text messaging) systems (Buu et al., 2017) or smartphone-based applications, where survey response options are immediately visible.

The population sampled may also affect compliance. Although younger populations may demonstrate greater proficiency with digital technology interfaces often used for EMA, a meta-analysis found an average compliance rate of 78.3% for youth participants in 42 mobile-EMA protocols (Wen et al., 2017). This rate is lower than the recommended 80% (Stone & Shiffman, 2002). Compliance rates in EMA studies with substance users vary across studies as a function of monitoring period and population of study. A meta-analysis of 126 EMA studies involving substance users enrolled in studies from 1998 to 2017 found an overall compliance rate of 75.06% (Jones et al., 2019). Other reports of EMA compliance involving substance users, and one specific to IVR technology, show compliance ranging from 50% to 70% (Buu et al., 2017; Kaminer et al., 2006). EMA studies involving substance use have shown that responsiveness to random prompts may be influenced by several factors (e.g., social context, affect, location) and that substance use behaviors can influence low compliance (Sokolovsky et al., 2014), suggesting that compliance may be further negatively impacted by substance co-use and severity of use (Messiah et al., 2011). Additionally, dispositional factors associated with inattentiveness and impulsivity, which are correlated with cannabis use (Cohn et al., 2015; Haas et al., 2018; O’Donnell et al., 2021), may also drive lower compliance rates. Studies have also shown that the degree of compensation for survey completion may impact survey compliance (Wrzus & Neubauer, 2021). Often, ways of addressing missing data in outcomes analyses include controlling for variables associated with missing data or imputing missing data based on either average ratings from the sample or from a participant’s own data (Cursio et al., 2019; Rendina et al., 2016). Reporting compliance rates and correlates thereof is important for determining the quality of a study, as this information could be used to determine inclusion in a systematic review, and could also assist other researchers determine whether IVR is a good fit for their study or selected population.

Given increasing prevalence of cannabis and tobacco co-use, particularly among young adults, and the popularity of studies utilizing EMA to understand substance use, additional research is needed to understand factors associated with daily EMA monitoring of co-use in younger age groups. To address this need, this study’s primary objective was to examine the prevalence and correlates of IVR-based EMA compliance and self-reported reactivity (e.g., behavioral and attitudinal change) to IVR monitoring in 97 young adult cannabis and tobacco co-users who completed IVR surveys three times a day for 28 consecutive days. Co-users were defined as individuals using cannabis >2 times a week in the past month and reporting “someday or every day” tobacco use. A secondary objective was to examine associations between IVR-based EMA compliance and self-reported reactivity, hypothesizing that respondents with higher compliance would report greater reactivity compared to respondents with lower compliance. A final objective was to examine whether self-reported reactivity would be associated with self-reported changes in tobacco and cannabis use behavior assessed from baseline to a post-IVR follow-up.

METHODS

Participants and Procedures

Data were collected in two large Northeastern cities in the U.S. between 2017 and 2019. Participants were 97 young adult cannabis and tobacco co-users who took part in an intensive longitudinal IVR-based EMA study about tobacco and cannabis use and co-use behavior. Participants were recruited via print and web-based advertisements and by word of mouth. Eligibility criteria included: 1) aged 18-24; 2) used cannabis ≥2 times a week in the past month; and 3) reported current “someday or every day” tobacco use (including e-cigarettes). Exclusion criteria were: 1) severe psychiatric disturbance; 2) potential for lethal alcohol consumption at least once in the past 3-months (as evidenced by BAC ≥0.20 based on reported drinks, gender, and weight); 3) dependence on substances other than alcohol, cannabis, caffeine, or nicotine; and 4) pregnant, planning to become pregnant, or breastfeeding.

After determining eligibility, participants completed a baseline session. After a brief training on the IVR system, participants engaged in a 28-day IVR-based EMA regimen in which they received calls at three random times per day to their phone (morning, midday, and evening), resulting in a total of 84 possible surveys. EMA responses were recorded using an IVR system. Following the 28-day period of EMA, participants completed a brief online assessment to query about satisfaction with and reactivity to EMA, as well as past 30-day tobacco and cannabis use behavior. Participants were compensated $25 for completing the baseline survey, $20/week for 4 weeks of IVR monitoring, an additional $1 for each random assessment completed (maximum of $3/day), and a bonus of $2/week for completing assessments 6 of 7 days or $5/week for completing assessments for all 7 days. Compensation totaled a maximum of $184 for completing the IVR surveys and $10 for completing the post-IVR survey at the 30-day follow-up. More details about the study methodology can be found here (Wilhelm et al., 2020). This study was approved by the Battelle Memorial Institute Institutional Review Board.

Materials Baseline Measures

Demographic Information

Basic demographic information was collected including age, sex, race/ethnicity, and employment status.

Cannabis Use

Participants reported the number of days they used cannabis in the past 30 days. Participants were also asked about average cannabis intoxication in the past 30 days (“On a typical day in the past 30 days, how high did you get when you used cannabis?”), where response options were on a scale from 1 to 10 with 10 being “the highest you’ve ever been.” Participants also reported their motivation to quit using cannabis by answering “How motivated are you to quit using cannabis right now?” Response options were measured on a 10-point scale with 1=“being not at all” and 10=“being highly motivated”. The Cannabis Use Disorder Identification Test-Revised (CUDIT-R) is an 8-item self-report measure that was used to assess likelihood of a cannabis use disorder (CUD) (Adamson et al., 2010). Scores ≥12 indicate probable CUD. The CUDIT-R has strong reliability and predictive validity with external diagnostic measures (Schultz et al., 2019).

Tobacco Use

Participants were asked to report the number of days in the past 30 days they used each of five different tobacco products (cigarettes, large cigars, little cigars/cigarillos, e-cigarettes, hookah/shisha/waterpipe). A sum variable was created capturing the number of tobacco products used in the past 30 days (range=0–5). Past 30-day cigarette smokers were also asked to report the number of cigarettes smoked per day (CPD). As a proxy for nicotine dependence, and consistence with other published work (Baker et al., 2007; Branstetter et al., 2020; Cohn, Rose, et al., 2019), participants were asked “How soon after you wake up do you use your first nicotine/tobacco product?”, with response options “within the first 5 minutes”, “6 to 30 minutes after waking”, “31–60 minutes after waking”, and “after 60 minutes”. The modified Contemplation Ladder (CL) assessed participants’ motivation to quit nicotine/tobacco using a 10-point scale where 1=“no thoughts of quitting” and 10=“taking action to quit” (Biener & Abrams, 1991). The CL has shown good convergent validity with other measures of motivation to change and predicts longer term readiness to quit smoking in samples of adults (McDermut & Haaga, 1998). Participants were asked about co-use behavior including “How long has it been since you last smoked part or all of a cigar/cigarillo with cannabis in it?” with response options “within the past 30 days”, “more than 30 days ago but within the past year”, and “more than a year ago.”

Alcohol Use

Participants were asked: “How many drinks of alcohol did you have per drinking episode in the past 30 days?”, with answer choices ranging from 0-10+ drinks per episode. Male participants who indicated ≥5 drinks per episode and female participants who indicated ≥4 drinks per episode were categorized as binge drinking in the past 30 days. Alcohol frequency in the past 30 days was assessed with the question, “During the past 30 days, on how many days did you use alcohol?„

Mental Health

Anxiety was measured using the 7-item Generalized Anxiety Disorder-7 (GAD-7) (Spitzer et al., 2006), which assesses current symptoms of an anxiety disorder. Participants used a scale of 0=“not at all” to 3=“nearly every day” to indicate how often they experienced each item in the past 2-weeks. A score of ≥8 indicates symptoms consistent with a generalized anxiety disorder (Kroenke et al., 2007). The GAD-7 has good reliability and validity (Löwe et al., 2008). Depression was measured using the 10-item Centers for Epidemiologic Studies Depression Scale-Revised (CESD-R) (Eaton et al., 2004). Participants indicated how often they experienced each item in the past week using a scale where 0=“rarely or none of the time (less than 1 day)” to 4=“all of the time (5–7 days)”. Scores ≥10 are indicative of current depression. The CESD-R has demonstrated high internal consistency, test-retest reliability, and external validity with other measures of mental health (Van Dam & Earleywine, 2011).

Personality Characteristics

The 4-item Brief Sensation-Seeking Scale (BSSS) was used to examine sensation-seeking personality disposition (Stephenson et al., 2003). Participants indicated the extent to which they agree or disagree with statements about their behavior or attitudes (e.g., “I would like to explore strange places”) with response options on a 5-point Likert scale (1=“strongly disagree” and 5=“strongly agree”). A sum score was created, where higher scores indicated greater sensation-seeking. The BSSS has shown good reliability and validity (Hoyle et al., 2002). The 8-item Barratt Impulsivity Scale (BIS) was used to measure impulsivity. Participants are asked to indicate how often, if ever, they engage in different behaviors or thoughts (e.g., “I plan tasks carefully”) with response options on a 4-point Likert scale (1=“rarely/never” and 4=“almost always/always”). A sum score was created where higher scores indicated greater levels of impulsivity. The 8-item BIS has shown good construct validity (Steinberg et al., 2013).

IVR Assessments

Participants were asked about cannabis and tobacco use since the last survey and assessed on current positive and negative mood, cannabis craving, any cannabis use and individual modes of cannabis use (e.g., blunt, spliff, bong, joint, edible, vaporizer, concenrates, etc), combined cannabis and tobacco use and type of tobacco product used (if applicable; e.g., cigarette, large cigar or little cigar, hookah, e-cigarette), subjective rating of cannabis intoxication, use of cannabis with others or alone, alcohol use (number of standard drinks consumed), and immediate negative perceived risks and benefits of cannabis use (e.g., getting into an argument, doing better on a task, drinking too much, feeling more creative, feeling more motivated to get things done, etc.) More detail about combined use with cannabis and tobacco are reported in Wilhelm et al. (2020). IVR surveys were programmed to occur at three random times a day (morning, midday, and evening) using an adaptive random prompting schedule corresponding to the sleep-wake cycle of each participant. No surveys were deployed within one hour of each other. IVR entries lasted approximately 5 minutes, were date- and time-stamped, and recorded immediately. For each survey, participants received a prompt (i.e., call) to their phone. If a participant delayed a prompt or did not answer, two additional follow-up prompts were sent, each 5 minutes apart, giving the participant a 15-minute completion window. After the third unaccepted prompt, the trial was recorded as missed.

Post-EMA Survey

The post-IVR survey was completed at the 1-month follow-up (e.g., immediately post-IVR). The survey was developed specifically for this study and has been used in previously published work (Cohn, Elmasry, et al., 2018).

Receptivity

Participants were asked: (1) “Did you feel that the daily phone calls took too much time?”, (2) “Were the questions easy to understand?”, (3) “Did it become easier and faster to complete the survey each day as time went on?”, and (4) “Did you find that the calls were disruptive to your regular schedule?” using a response scale of 0=“Not at all”, 1=“Slightly”, 2=“Moderately”, 3=“Very Much”, and 4=“Extremely”.

Perceived Reactivity

Participants were asked about increased awareness of their behavior and purposeful change to their behavior with the following items: “To what extent did you feel that the daily phone calls may have caused you to be more aware of your behavior?” and “Did you find that you purposely started to make changes to your behavior because of the daily monitoring?” Response options for these two questions were 0=“Not at all”, 1=“Slightly”, 2=“Moderately”, 3=“Very Much”, and 4=“Extremely”. Participants were then asked: “Did you begin to notice any behaviors more than before, and if so, which ones? Please select all that apply” with 15 choices including “Cannabis use” and “Smoking cigarettes or other nicotine/tobacco use.” Lastly, an item about purposeful behavior change asked: “Which behaviors did you purposely make changes to? Select all that apply” with 13 choices including “Using cannabis less often”, “Smoking cigarettes or using other nicotine/tobacco less often”, and “None of the above”.

Substance Use

At the follow-up, participants were asked to report the number of days they used the following products in the past 30 days: cigarettes, cigars, little cigars/cigarillos/bidis, e-cigarettes, hookah, alcohol, and cannabis.

Data Analysis

Descriptive statistics and repeated measures Analysis of Variance (ANOVAs) tests were used to examine the prevalence of, and differences in overall compliance rates by time of day (morning, midday, evening) and week of assessment (weeks 1-4). Mauchley’s test of sphericity was significant for the analysis of weekly differences in compliance (Chi-square=34.04(5), p<.001), indicating a violation of the assumption of equal variances. The Greenhouse-Geisser Epsilon of 0.791 was greater than 0.75, so the Huynh-Feldt corrected F-test was reported for weely differences in compliance (Verma, 2015). Compliance was defined as the number of calls completed in a given time period (i.e., overall, morning, midday, evening, or week) divided by the number of total possible calls during that same time period.

Using the same aproach as Simpson et al. (2012), the sample was dichotomized into high and low frequency responders. Participants who completed ≥70% of the calls were considered high frequency responders, and those who completed <70% of the calls were considered low frequency responders. ANOVAs and chi-square tests were then performed to examine differences in high and low frequency responders on baseline factors (demographic characteristics, cannabis/tobacco use behavior, alcohol use, mental health, and personality factors).

Next, differences in receptivity and self-reported reactivity to the IVR calls were examined across high and low frequency responders using ANOVAs. Lastly, four hierarchical regression models were used to examine the predictive utility of self-reported reactivity on the frequency of past 30-day (1) cigarette and (2) cannabis use assessed at the post-IVR survey (i.e., 1-month follow-up), controlling for baseline reports of the outcome. Specifically, the self-reported reactivity items used were: increased awareness of cigarette smoking, increased awareness of cannabis use, purposeful behavior change of cigarettes, and purposeful behavior change of cannabis. Reactivity items were examined in separate models. Step 1 of the model included the baseline report of the outcome, and Step 2 included the requisite IVR index of self-reported reactivity. SPSS 27.0 was used for all analyses.

RESULTS

Survey Compliance by Time of Day and Day of Week

Overall, participants completed 4,507 (55.3%) of the 8,148 total possible IVR surveys. Compliance for morning, midday, and evening was 54.6%, 56.8%, and 54.5% respectively. There were no differences in compliance by time of day [F(2, 192)=2.56, p=.080, partial η²=.026]. Compliance by assessment week (Week 1, Week 2, Week 3, Week 4) was 70.3%, 56.6%, 51.2%, and 43.3% respectively. Compliance signficantly decreased across the 4 weeks of IVR-based EMA [F(2.44, 233.85)=62.75, p < .001, partial η²=.395].

Sample Characteristics and Differences across High and Low Frequency Responders

Sample characteristics for the full sample and across high and low frequency responders are shown in Table 1. A third of the sample (36.1%; n=35) were high frequency responders (completed ≥70% of IVR calls). On average, participants were 21.32 years old (SD=1.90), and the majority were male, White, and employed. At baseline, participants consumed cannabis most days out of the month (24.67 days; SD=7.98), reported moderate levels of cannabis intoxication when they used cannabis (M=6.55 out of 10; SD=1.55), and reported low motivation to quit using cannabis (M=4.37 out of 10; SD=2.34). Most also reported symptoms consistent with a Cannabis Use Disorder (CUD) (61.9%). Participants used alcohol 6.27 days (SD=6.64) in the past 30 days with an average of 3.01 drinks (SD=2.35) per drinking episode. Nearly a quarter of the sample (24.7%) reported binge drinking in the past 30 days.

Table 1.

Baseline Sample Characteristics and Differences Across High and Low Frequency IVR Responders

		TOTAL	HIGH FREQUENCY RESPONDERS (n=35)	LOW FREQUENCY RESPONDERS (n=62)
	M/n	SD/%	M/n	SD/%	p
Age (M, SD)	21.32	1.90	21.79 (1.90)	21.05 (1.85)	.067
Sex
Female	41	42.3%	13 (31.7%)	28 (68.3%)	.470
Male	54	55.7%	21 (38.9%)	33 (61.1%)
Race/Ethnicity
Non Hispanic Black	20	20.6%	6 (30.0%)	14 (70.0%)	.827
Non Hispanic White	48	49.5%	17 (35.4%)	31 (64.6%)
Non Hispanic Other	19	19.6%	8 (42.1%)	11 (57.9%)
Hispanic	9	9.3%	4 (44.4%)	5 (55.6%)
Employment Status
Employed (full or part time)	50	51.5%	16 (32.0%)	34 (68.0%)	.501
Unemployed/Disabled	18	18.6%	6 (33.3%)	12 (66.7%)
Student	29	29.9%	13 (44.8%)	16 (55.2%)
Cannabis use	M	SD	M (SD)	M (SD)
Days cannabis used in past 30 days	24.67	7.98	24.03 (8.48)	25.03 (7.74)	.555
Perceived cannabis intoxication in past 30 days	6.55	1.55	6.66 (1.19)	6.48 (1.73)	.599
Motivation to quit cannabis	4.37	2.34	3.75 (2.34)	4.70 (2.31)	.263
	n	%	n (%)	n (%)
Cannabis dependence (CUDIT-R ≥ 12)	60	61.9%	19 (31.7%)	41 (68.3%)	.249
Tobacco product use	M	SD	M (SD)	M (SD)
Number tobacco products used in past 30 days	2.27	1.11	2.26 (1.40)	2.27 (0.93)	.943
Number days smoked cigarette in past 30 days	16.65	12.43	15.39 (12.41)	17.38 (12.53)	.546
Cigarettes per day	5.49	6.14	5.91 (7.04)	5.24 (5.63)	.691
Motivation to quit nicotine/tobacco	5.80	2.56	5.86 (2.50)	5.77 (2.61)	.875
Nicotine dependence	n	%	n (%)	n (%)
Tobacco use within 5 minutes of waking	15	15.5%	3 (20.0%)	12 (80.0%)	.373
Tobacco use within 6-30 minutes of waking	24	24.7%	7 (29.2%)	17 (70.8%)
Tobacco use within 31-60 minutes of waking	15	15.5%	6 (40.0%)	9 (60.0%)
Tobacco use after 60 minutes of waking	42	43.3%	18 (42.9%)	24 (57.1%)
Alcohol use	M	SD	M (SD)	M (SD)
Drinks per drinking episode in past 30 days	3.01	2.35	2.77 (2.37)	3.15 (2.35)	.455
Number days used alcohol in past 30 days	6.27	6.64	7.54 (8.14)	5.55 (5.57)	.157
	n	%	n (%)	n (%)
Binge drinking in past 30 days	24	24.7%	7 (29.2%)	17 (70.8%)	.434
Mental health
Anxiety (GAD-7 ≥ 8)	36	37.1%	12 (33.3 %)	24 (66.7%)	.665
Depression (CESD-R ≥ 10)	45	46.4%	15 (33.3%)	30 (66.7%)	.600
Personality characteristics	M	SD	M (SD)	M (SD)
Sensation-seeking (BSSS-4)	14.96	3.20	15.09 (3.48)	14.89 (3.06)	.771
Impulsivity (BIS-8)	15.92	4.33	15.97 (4.67)	15.89 (4.17)	.927

Note. GAD-7, seven-item Generalized Anxiety Disorder Scale; CESD-R, Center for Epidemiologic Studies Depression Scale-Revised (CESD-R); BSSS-4, four-item Brief Sensation-Seeking Scale; BIS, eight-item Barratt Impulsivity Scale

*** p < .001. ** p < .01. * p < .05

In terms of individual tobacco product use in the past 30 days at baseline (not shown in Table 1), 65% reported cigarette use, 62.9% reported e-cigarette use, 59.8% reported little cigar/cigarillo use, 28.9% reported hookah use, and 10.3% reported large cigar use. On average, particants used 2.3 tobacco products in the past 30 days (SD=1.1). Participants reported smoking cigarettes slightly more than half the days in the past month (16.65 days; SD=12.43) and those who reported smoking a cigarette at least once in the past 30 days, smoked an average of 5.49 cigarettes per day. Average motivation to quit nicotine/tobacco was (M=5.80 out of 10; SD=2.56). In terms of nicotine dependence, 15.5% of past 30-day cigarette smokers reported using tobacco within 5 minutes of waking up indicating high dependence. 73.2% of participants reported smoking part or all of a cigar/cigarillo with cannabis in it within the past 30 days. In terms of mental health factors, 46.4% of the sample reported symptoms consistent with current depression and 37.1% reported symptoms consistent with anxiety.

High and low frequency responders did not differ significantly on any baseline factors (all p’s>.05).

IVR Receptivity and Self-Reported Reactivity

A total of 84 participants (87%) completed the post-IVR survey. A significantly higher proportion of completers reported symptoms consistent with anxiety compared to non-completers (41.7% vs 7.7%), χ²(1)=5.57, p=.015. No other baseline factors were associated with post-IVR survey completion.

Figure 1 shows how respondents answered receptivity items. When asked if the daily phone calls took too much time, 66% of participants answered “not at all” or “slightly” (Panel A). The majority (83%) reported that the surveys were “very much” or “extremely” easy to understand (Panel B). When asked whether it became easier and faster to complete the surveys as time went on, 73% said “very much” or “extremely” (Panel C), and when asked if the calls were disruptive to their regular schedule, 52.6% answered “not at all” or “slightly” (Panel D).

Figure 1.

Proportion of Respondents Reporting that IVR Surveys were Burdensome (A), Easy to Understand (B), Became Easier to Take Over Time (C), and Disruptive (D)

ANOVA tests examining differences in receptivity and self-reported reactivity ratings among high and low frequency responders are shown in Table 2. Low frequency responders reported that the daily surveys took too much time relative to high frequency responders [F(1,82)=6.48, p=.019]. Relative to low frequency responders, high frequency responders reported that it became easier and faster to complete the surveys over time [F(1, 82)=3.82, p=.025] and that calls were less disruptive [F(1, 82)=18.62, p<.001].

Table 2.

Receptivity and Reactivity to IVR among High and Low Rrequency Responders

	Full Sample	High Frequency (n=35)	Low Frequency (n=62)	p
	M (SD)	M (SD)	M (SD)
Receptivity
Took too much time	1.07 (1.10)	0.74 (0.9)	1.31 (1.2)	.019
Easy to understand	3.55 (0.63)	3.49 (0.7)	3.59 (0.5)	.449
Became easier and faster	3.26 (0.88)	3.51 (0.7)	3.08 (1.0)	.025
Calls were disruptive	1.50 (1.18)	0.94 (0.8)	1.9 (1.2)	<.001
Reactivity
Increased awareness	2.10 (1.13)	1.94 (1.2)	2.20 (1.1)	.297
Purposeful behavior change	0.89 (0.99)	0.94 (1.1)	0.86 (0.9)	.699

Note. Response options range from 0 to 4 with 0 = “Not at all”, 1 = “Slightly”, 2 “Moderately”, 3 = “Very much”, and 4 = “Extremely” Significant differences bolded for emphasis.

In terms of self-reported reactivity, high and low frequency responders did not differ on increased awareness of behavior [F(1, 82)=1.10, p=.297] or degree of purposeful behavior change [F(1, 82)=0.150, p=.699]. Figure 2 shows the proportion of respondents who reported increased awareness and purposeful behavior change of nicotine/tobacco and cannabis use in response to IVR assessments. With respect to increased awareness, 11.3% (n=11) reported increased awareness of nicotine/tobacco use and 18.6% (n=18) reported increased awareness of cannabis use; while16.5% (n=16) reported purposefully using nicotine/tobacco less often, and 19.6% (n=19) reported purposefully using cannabis less often.

Figure 2.

Self-Reported Increased Awareness and Purposeful Change to Nicotine/Tobacco and Cannabis Use Behavior from IVR Monitoring

Associations of Self-Reported Reactivity to Behavior Change

Table 3 shows results of hierarchical linear regression models that examined associations of indices of self-reported reactivity (increased awareness and purposeful behavior change) with actual behavior change outcomes. Only purposely using cannabis less was associated with fewer cannabis use days at the post-IVR survey (B= −0.29, p>.001), controlling for baseline levels of cannabis use. No other significant associations were found.

Table 3.

Hierchical Regression Models Results Of The Associations Between Indices Of IVR-Based EMA Reactivity With Post-IVR Past 30-Day Cigarette And Cannabis Use

	Past 30-day cigarette use		Past 30-day cannabis use
	B	p	B	p
Increased awareness of nicotine/tobacco use	0.01	.933	-	-
Purposely used nicotine/tobacco less	−0.08	.462	-	-
Increased awareness of cannabis use	-	-	0.12	.124
Purposely used cannabis less	-	-	-0.29	>.001

Note. Models controlled for baseline levels of the outcome in Step 1. Significant associations bolded for emphasis.

DISCUSSION

This study is the first to examine IVR-based EMA compliance among young adult tobacco and cannabis co-users and adds to the work examining factors associated with EMA compliance, receptivity, and perceived reactivity. The compliance rate for this study was 55.3%, which is lower than the recommended 80% (Stone & Shiffman, 2002), but falls in the 50–70% range reported for EMA studies involving substance use in similar samples of young adults or adolescents (Buu et al., 2017; Kaminer et al., 2006). In this study, compliance decreased across all four weeks, consistent with results from other EMA studies (Battista et al., 2015; Hoeppner et al., 2014). This study characterized 36.1% of the sample as high frequency responders for having completed ≥70% of the calls and found no differences between high and low frequency responders on any demographic variables or baseline measures of cannabis use, tobacco use, nicotine dependence, alcohol use, mental health, and personality characteristics. A few studies have found associations between low EMA compliance and substance use factors such as alcohol use and smoking (Litt et al., 1998; Sokolovsky et al., 2014), while meta-analyses report little evidence of associations between low EMA compliance and study design, participant characteristics, or substance type (Jones et al., 2019; Wrzus & Neubauer, 2021). A meta-analysis of 126 EMA studies involving substance use reported a pooled compliance that was lowest for studies assessing cannabis (66.16% pooled compliance) compared to studies primarily assessing tobacco (77.79% pooled compliance) or alcohol (76.36% pooled compliance) (Jones et al., 2019). While our findings did not show any specific impact of baseline cannabis use frequency on compliance, the sample’s overall low compliance may be because the sample consisted of regular cannabis users. Mental health conditions, such as susbstance use disorder, may compromise one’s ability to self-regulate thus impacting compliance. A variety of unmeasured factors associated with cannabis use (e.g., use motives, use consequences) may also have influenced overall compliance. Ways of improving overall compliance include offering financial incentives commensurate with time and burden associated with surveys and offering incentive escalation over time.

In terms of receptivity, the majority of participants reported that the surveys were easy to understand and not burdensome. Not surprisingly, a greater proportion of high frequency responders reported high receptivity to the surveys. It could be that participants who perceive surveys as becoming easier to complete over time may be more likely to continue completing surveys. This also suggests that participants who perceived greater burden over time are less likely to complete surveys over time. Nearly two thirds of participants (72.2%) reported increased awareness of at least one behavior in response to IVR assessments, and just over half (56.7%) reported purposefully changing at least one behavior. Regression models showed that purposely decreasing cannabis use was correlated with lower past 30-day cannabis use behavior at one month follow-up. This suggests that behavior monitoring via EMA has some relationship to changes in behavior over time, consistent with other studies that used-EMA monitoring (Cohn, Elmasry, et al., 2018; Gass et al., 2021). While observational studies using EMA are not intentionally attempting to influence change in behavior, this may be an unintended consequence of using frequent assessments. Plans to mitigate reactivity in EMA in observational studies should be considered when these methods are applied (Barta et al., 2012). These include examining response-shifts over time, reducing survey time and frequency, and incorporating post-EMA surveys of reactivity for covariate analyses. On the other hand, results suggest self-monitoring can be an effective tool for behavior change, even for cannabis use. Mobile health EMA technology, such as smartphone-based apps, has been utilized successfully as a treatment modality for a variety of substance use disorders (Businelle et al., 2016; Sherman & McRae-Clark, 2016). The increasing popularity of mobile health interventions offers promise as a potential treatment for cannabis and tobacco co-use. Similar principles of mobile health treatments for substance use could apply to the “treatment” of low compliance in EMA studies, such asintervening in the moment when a survey prompt has been missed or delayed, and providing daily feedback about progress.

Limitations

There were several limitations of this study worth noting. First, the study utilized IVR, despite smartphpone-based apps becoming increasingly popular. Second, nicotine dependence was measured using a 1-item assessment. While this has been shown to be a robust proxy for nicotine dependence, (Baker et al., 2007; Branstetter et al., 2020) the item was broad and not specific to any particular tobacco product. Third, in an attempt to minimize participant burden and reduce survey completion time, questions assessing late night use of cannabis were not asked despite sleep disturbance being a common reason for use. It is possible that poor sleep could have had an impact on compliance. Additionally, participants were asked to elucidate specific reasons for survey non-compliance and data was not normally distributed potentially impacting Type 1 error. Finally, generalizability of study findings is limited given that this study did not recruit individuals across the age continuum or non-users of both tobacco and cannabis. It is possible non-users may have higher compliance rates than current substance users, but study methodology prevents this determination.

Conclusion

Given the growing number of states with legal cannabis and high rates of the co-use of cannabis with tobacco among young adults, it is imperative to capture changes in co-use in this population. This study evaluated parameters of IVR-based EMA compliance in young adult tobacco and cannabis co-users, to elucidate factors that could be targeted in future studies using similar methodologies, in an effort to improve compliance. We found no differences in baseline characteristics between high and low frequency responders. Even with several assessments per day for a month, findings show high IVR receptivity and suggest that self-reports of behavior and attiduinal change occur, and may predict some modest behavior change after monitoring. Measures of reactivity to daily monitoring highlight the effectiveness of mobile health technologies to improve substance use. Ensuring high compliance could also improve the effectiveness of such technology.