Abstract
Background
The COVID-19 pandemic disrupted access to treatment for substance use disorders (SUDs), while alcohol and cannabis retail sales increased. During the pandemic, we tested a tailored digital health solution, Woebot-SUDs (W-SUDs), for reducing substance misuse.
Methods
In a randomized controlled trial, we compared W-SUDs for 8 weeks to a waitlist control. U.S. adults (N = 180) who screened positive for substance misuse (CAGE-AID>1) were enrolled June–August 2020. The primary outcome was the change in past-month substance use occasions from baseline to end-of-treatment (EOT). Study retention was 84%. General linear models tested group differences in baseline-to-EOT change scores, adjusting for baseline differences and attrition.
Results
At baseline, the sample (age M = 40, SD = 12, 65% female, 68% non-Hispanic white) averaged 30.2 (SD = 18.6) substance occasions in the past month. Most (77%) reported alcohol problems, 28% cannabis, and 45% multiple substances; 46% reported moderate-to-severe depressive symptoms. Treatment participants averaged 920 in-app text messages (SD = 892, Median = 701); 96% of completed lessons were rated positively; and 88% would recommend W-SUDs. Relative to waitlist, W-SUDs participants significantly reduced past-month substance use occasions (M = −9.1, SE = 2.0 vs. M = −3.3, SE = 1.8; p = .039). Secondary substance use and mood outcomes did not change significantly by group; however, reductions in substance use occasions correlated significantly with increased confidence and fewer substance use problems, cravings, depression and anxiety symptoms, and pandemic-related mental health effects (p-value<.05).
Conclusions
W-SUDs was associated with significant reductions in substance use occasions. Reduction in substance use occasions was associated with better outcomes, including improved mental health. W-SUDs satisfaction was high.
Keywords: COVID-19 pandemic, Artificial intelligence, Substance-related disorders, Randomized controlled trial, Relational conversational agent
1. Background
The 2016 Surgeon General’s Report on Facing Addiction in America warned that community-wide disasters can disrupt treatment services and/or increase stress, precipitating relapse or heavier substance use (U.S. Department of Health and Human Services, 2016). Before the COVID-19 pandemic, addiction treatment in the U.S. reached less than one in five individuals who met diagnostic criteria for a substance use disorder (SUD) (Substance Abuse and Mental Health Services Administration, 2019). The COVID-19 pandemic disrupted treatment access further, while retailers selling alcohol and, in some states, cannabis, were deemed essential businesses. In March 2020, compared to the year prior, alcohol retail sales increased 54% and online alcohol sales increased 262% (The Nielsen Company, 2020). Surges in cannabis sales were recorded in March 2020, and sales growth for the year was 40% stronger than 2019 (Vangst et al., 2020).
The pandemic’s extended physical distancing demands coupled with the many social, medical, political, racial/ethnic and economic stressors of 2020 could escalate substance misuse, relapse, and the development of SUDs in at-risk individuals (Clay and Parker, 2020). In late June 2020, 13% of U.S. adults reported initiating or increasing substance use to cope with pandemic-related stress (Czeisler et al., 2020). Demands during and following the pandemic may overwhelm already strained addiction treatment services.
Digital health applications (apps) can reduce barriers to traditional SUD treatment including stigma, costs and inadequate insurance coverage, time demands, transportation needs, lack of access to qualified providers, and challenges navigating complex treatment systems (Giroux et al., 2017). Preliminary evidence suggests that digital SUD interventions can reduce substance misuse (Boumparis et al., 2019; Giroux et al., 2017). Scalable digital SUD interventions could lessen the population-level burden of SUDs.
A novel approach for scaling therapeutic interactions without requiring human involvement is the use of artificial intelligence (AI). Programmed therapeutic relational agents can deliver a coach-like or sponsor-like experience for in-the-moment treatment delivery (Vaidyam et al., 2019). Evidence suggests that people are more likely to disclose personal information when they believe the interactions are computer- rather than human-monitored (Lucas et al., 2014); and a strong therapeutic alliance can develop in the absence of face-to-face contact (Cook and Doyle, 2002), even with a non-human app (Berry et al., 2018). While most mental health apps have high drop-off (Baumel et al., 2019; Torous et al., 2018), a meta-analysis found that conversational text-based agents may increase engagement and enjoyment in digitized mental health care (Vaidyam et al., 2019). Preliminary research indicates text-based presentation results in higher program adherence than verbal presentation (Tielman et al., 2017).
In an 8-week, single-group pre/post evaluation, we recently evaluated Woebot for Substance Use Disorders (W-SUDs) (Prochaska et al., 2021). W-SUDs was adapted from Woebot, a therapeutic relational agent found to decrease depressive symptoms (Fitzpatrick et al., 2017). Delivered via a smartphone app, W-SUDs provides psychoeducation in cognitive behavioral therapy (CBT)–based behavior change tools and encourages mood tracking and behavioral pattern insight. Participants were found to engage with W-SUDs regularly, reporting high acceptability and affective bond formation. From pre- to post-treatment, significant reductions were reported in substance use occasions, hazardous alcohol and drug use, and cravings to use substances, while confidence to resist substance use urges increased (Prochaska et al., 2021). Depression and anxiety symptoms significantly declined. Given these encouraging findings, we aimed to evaluate the efficacy of W-SUDs relative to a waitlist group in a randomized controlled trial. The trial was conducted in summer 2020, when the need for novel in-hand treatment options was amplified due to the COVID-19 pandemic.
2. Methods
2.1. Study design
We evaluated W-SUDs in an 8-week, two-group randomized controlled trial with a waitlist comparison condition. The primary outcome was change in past-month substance use occasions self-reported at baseline and end-of-treatment (EOT). Secondary outcomes were changes in substance use problems, craving intensity, confidence to resist substance use urges, mood symptoms (depression, anxiety), pain, and pandemic-related mental health effects. Intervention engagement data were collected from the W-SUDs app. Acceptability was assessed within the app and in the EOT survey.
2.2. Recruitment and randomization
Study procedures were approved by Stanford Medicine’s Institutional Review Board. Participants were recruited June 25 to August 18, 2020 via Qualtrics Research Services, Stanford listservs, Facebook, and word-of-mouth. Recruitment materials sought adults with substance use concerns. Informed consent was required for eligibility screening and study participation.
Inclusion criteria were ages 18–65 years, residing in the U.S., scoring >1 on the CAGE-AID (Cut down, Annoyed, Guilty, Eye opener-Adapted to Include Drugs) (Brown and Rounds, 1995), owning a smartphone for accessing W-SUDs, available for the 8-week study, providing an email address, and English literacy. A cut-point of >1 on the CAGE-AID has a sensitivity of 70% and specificity of 85% for identifying individuals with SUDs (Brown and Rounds, 1995). Study exclusion criteria were prior Woebot use, anticipated pregnancy during the study period, history of severe alcohol or drug-related medical problems (e.g., delirium tremens, seizure, liver disease, hallucinations), opioid overdose requiring Narcan (naloxone), current opioid misuse without medication-assisted treatment, or past-year attempted suicide. To maximize study retention, we obtained participants’ phone numbers and provided up to U.S.$75 in Amazon gift cards for completing study assessments.
Based on power calculations, with a two-sided rejection region, alpha level of 0.05, power of 80%, projected effect size of Cohen’s d = 0.50, and 20% attrition, the target sample size was N = 80 per group or 160 participants. Fig. 1 shows the study consort diagram, which is missing some exclusion information lost due to a clerical error. Based on captured data, leading reasons for study exclusion were a CAGE-AID score of 0 or 1 (n = 1149) and being unable to commit to the 8-week study 2 (n = 401). All 193 eligible individuals provided informed consent for study participation; 189 completed the baseline survey; however, 5 were duplicates and 4 provided invalid contact information. In total, 180 participants were randomized to W-SUDs (n = 88) or the waitlist control (n = 92) group. Randomization was stratified on reported substance use problems on the SIP-AD measure (Blanchard et al., 2003), described below.
Fig. 1.
Woebot for Substance Use Disorders (W-SUDs) Randomized Controlled Trial Consort Diagram.
2.3. Study conditions
2.3.1. W-SUDs intervention
Woebot is a therapeutic relational agent that delivers CBT in the format of brief, daily text-based conversations through its own native app on iPhone and Android devices (Fitzpatrick et al., 2017). The app onboarding process introduces Woebot, explains the intended use of the device, how data are treated, and limitations of the program (e.g., that it is not a crisis service). Participants are encouraged to check-in with Woebot daily and informed that check-ins will invite weekly tracking of their mood and ratings of substance use craving and pain. Daily push notifications prompt users to check in. Woebot is introduced as a guided self-help coach and explicitly not human. Participants are educated that Woebot will offer emotional support, daily lessons, and tools for reducing substance use and manage cravings, all via the medium of text-based conversations. Conversations with Woebot are responsive in real-time to users’ inputs and crafted to offer the tool and or lesson for participants’ moment of need. After the app onboarding process, Woebot hosts the first conversation with the participant and demonstrates app navigation and tool application. Woebot was adapted for the treatment of SUDs (W-SUDs), drawing upon motivational interviewing principles, mindfulness training, dialectical behavior therapy, and CBT for relapse prevention (Prochaska et al., 2021). The W-SUDs intervention offers 66 psychoeducational lessons and psychotherapeutic skills and was designed as an 8-week treatment. Brief intervention can minimize dropout, a problem common to SUD treatment (Brorson et al., 2013). Given the sustained nature of the COVID-19 pandemic, content was added to W-SUDs to acknowledge pandemic-related stressors and isolation (Fig. 2 ).
Fig. 2.
Sample Screen Shots of Woebot for Substance Use Disorders (W-SUDs).
2.3.2. Waitlist control
Participants randomized to the waitlist were provided access to W-SUDs after completing the 8-week study.
2.4. Assessments
Study assessments were collected at baseline and EOT and were self-reported online via Qualtrics. Baseline demographic items were sex, race/ethnicity, age, marital and employment status, and zip code. Ten lifetime psychiatric diagnoses were assessed with a write-in option for others. Prior therapy experience (ever and current) and current psychiatric medication use were assessed. Engagement in SUD treatment in the past month at baseline and past 2 months at EOT (i.e., during the study period) was reported for self-help groups, outpatient, intensive outpatient, residential/inpatient, and medications to reduce substance use. Pandemic-related life disruptions were queried including changes in employment and income; engagement in COVID-19 precautionary behaviors in the past 2 weeks (4-point scale: never to most of the time, see Table 1 ) (Lavoie and Bacon, 2020); difficulty accessing medical treatment or getting necessities (yes/no) (Lang, 2020); and COVID-19 diagnosis.
Table 1.
Baseline Participant Demographic, Mental Health, and Substance Use Characteristics by Group.
| W-SUDS Group (N = 88) |
Waitlist Control Group (N = 92) |
|||
|---|---|---|---|---|
| Variable | M (SD), Range | % (n) | M (SD), Range | % (n) |
| Age in Years | 40.8 (12.1), 19-65 | 39.8 (11.2), 18-64 | ||
| Biological Sex | ||||
| Female | 65.9% (58) | 64.1% (59) | ||
| Male | 34.1% (30) | 35.9% (33) | ||
| Hispanic/Latinx Ethnicity | 13.6% (12) | 13.0% (12) | ||
| Race | ||||
| White | 79.5% (70) | 76.1% (70) | ||
| Black/African American | 12.5% (11) | 12.0% (11) | ||
| Asian American | 4.5% (4) | 7.6% (7) | ||
| American Indian or Alaska Native | 2.3% (2) | 2.2% (2) | ||
| Multiracial | 1.1% (1) | 2.2% (2) | ||
| Marital Status | ||||
| Married/Cohabitating/Partnered | 56.8% (50) | 55.4% (51) | ||
| Divorced/Separated/Widowed | 9.1% (8) | 9.8% (9) | ||
| Never Married/Single | 34.1% (30) | 34.8% (32) | ||
| Educational Degree | ||||
| High School Degree | 29.5% (26) | 29.3% (27) | ||
| College Degree | 48.9% (43) | 53.3% (49) | ||
| Graduate or Post-graduate Degree | 21.6% (19) | 17.4% (16) | ||
| Employment Status Prior to the Pandemic§ | ||||
| Employed Part or Full-Time | 75.0% (66) | 76.9% (70) | ||
| Unemployed (homemaker, student, retired) | 25.0% (22) | 23.1% (21) | ||
| Prior Therapy Experience | ||||
| Never in Therapy | 47.7% (42) | 55.4% (51) | ||
| In Therapy Formerly | 37.5% (33) | 30.4% (28) | ||
| Currently in Therapy | 14.8% (13) | 14.1% (13) | ||
| Psychiatric Medications CurrentlyPD | 29.5% (26) | 31.5% (29) | ||
| Past 30 Day SUD Treatment | ||||
| Self-help groups | 13.0% (12) | 11.4% (10) | ||
| Outpatient Services | 13.0% (12) | 12.5% (11) | ||
| Intensive Outpatient / Residential | 1.7% (3) | 0.6% (1) | ||
| Past 30 Day Medication Use for SUD Treatment | 5.7% (5) | 8.7% (8) | ||
| Lifetime Mental Health Conditions# | ||||
| Unipolar Depression | 30.7% (27) | 26.1% (24) | ||
| Bipolar/Manic Depression | 8.0% (7) | 14.1% (13) | ||
| Anxiety Disorder | 43.2% (38) | 37.0% (34) | ||
| Other Mental Health Disorder | 37.5% (33) | 40.2% (37) | ||
| Multiple Diagnoses | 39.8% (35) | 38.0% (35) | ||
| No History of Mental Illness | 37.5% (33) | 43.5% (40) | ||
| Depressive Symptoms PHQ–8, possible range 0-24 | 8.4 (5.6), 0-20 | 9.3 (6.4), 0-24 | ||
| 10+ Moderate/Severe (%) | 45.5% (40) | 46.7% (43) | ||
| Anxiety Symptoms GAD 7-Anxiety, possible range 0-21 | 7.8 (5.7), 0-21 | 8.0 (6.2), 0-21 | ||
| 10+ Moderate/Severe (%) | 37.5% (33) | 41.3% (38) | ||
| Pain Past 7 day§, possible range 0-100 | 30.3 (26.5), 0-93 | 33.2 (29.1), 0-100 | ||
| Pain Past 30 day Interference Moderate-Extreme (%) | 23.9% (21) | 34.8% (32) | ||
| Primary or Secondary Problematic Substance | ||||
| Alcohol | 81.8% (72) | 72.8% (67) | ||
| Cannabis | 22.7% (20) | 33.7% (31) | ||
| Stimulants/Cocaine | 8.0% (7) | 15.2% (14) | ||
| Other (e.g., Heroin, Hallucinogens, Inhalants) | 13.6% (12) |
15.2% (14) |
||
| Past 30 Days of Substance Use | M (SD) daysM |
n any use (%) |
M (SD) daysM |
n any use (%) |
| Alcohol | 16.8 (9.4) | 79 (89.8%) | 16.8 (10.1) | 74 (80.4%) |
| Cannabis | 17.6 (10.3) | 39 (44.3%) | 18.3 (11.4) | 59 (64.1%) |
| Cocaine | 9.0 (5.6) | 3 (3.4%) | 9.7 (8.5) | 9 (9.8%) |
| Prescription stimulants | 25.2 (6.9) | 5 (5.7%) | 11.7 (11.1) | 12 (13.0%) |
| Methamphetamine | 17.0 (12.7) | 2 (2.3%) | 15.5 (20.5) | 2 (2.2%) |
| Inhalants | 0 | 0 (0.0%) | 6.0 (6.1) | 4 (4.3%) |
| Sedatives | 8.1 (7.2) | 17 (19.3%) | 11.9 (12.2) | 18 (19.6%) |
| Hallucinogens | 5.0 (2.8) | 2 (2.3%) | 2.0 (2.0) | 4 (4.3%) |
| Street Opioids | 3.5 (2.1) | 2 (2.3%) | 3.0 | 1 (1.1%) |
| Prescription Opioids | 13.9 (11.5) | 7 (8.0%) | 16.7 (11.7) | 9 (9.8%) |
| Past-Month Substance Use OccasionsS | 27.9 (16.8), 0-68 | 32.4 (20.0), 0-90 | ||
| Drug Abuse Screening Test,* possible range 0-10 | 1.3 (1.8), 0-8 | 2.2 (2.1), 0-8 | ||
| 3+ Moderate/Severe (%) | 21.6% (19) | 41.3% (38) | ||
| Short Inventory of Problems – Alcohol and Drugs (SIP-AD), possible range 0-45 |
11.9 (10.8), 0-41 |
12.9 (10.4), 0-45 |
||
| CAGE-AID, possible range 2-4* | 2.8 (0.8), 2-4 | 3.1 (0.8), 2-4 | ||
| Confidence to Resist Urges, possible range 0-100% | 57.6 (23.8), 3.1-100 | 61.3 (21.8), 13.6-100 | ||
| Cravings Past 7 days, possible range 0-4 | 1.5 (1.1), 0-4 | 1.5 (1.2), 0-4 | ||
NAParticipants responding “not applicable” are treated as missing.
MMean days of use are calculated among those who report any use of that substance in the past 30 days.
§N = 179 reporting.
SCalculated by summing days of past-month use of 10 substances.
PDOnly asked of participants who reported a psychiatric diagnosis.
#Categories are not mutually exclusive.
*p < .05 significant group differences on CAGE-AID score at screening and DAST-10 score at baseline.
The specified primary outcome was the change from baseline to 8-weeks (EOT) in substance use occasions reported for the past 30 days. At baseline and EOT, the number of days used in the past 30 days was assessed for each of the following: alcohol, cannabis, cocaine, prescription stimulants, methamphetamine, inhalants, sedatives or sleeping pills, hallucinogens, street opioids, and prescription opioids. Because participants used different substances and not all drank alcohol, days of reported alcohol use and days of use for each assessed drug were summed to reflect past-month substance use occasions. Past-month substance use occasions could exceed 30 if individuals used more than one substance on a day. Previously, this measure was found sensitive to treatment effects (Prochaska et al., 2021). Prior research has supported the validity of frequency measures of substance use consumption, which provide a continuous measure of the severity of harmful, problematic use (Moss et al., 2012) and when combined across substances, appropriately take into account their collective contributions to substance-related problems (Lennox et al., 2006). From the list, participants identified their primary and secondary (if applicable) substance(s) of abuse.
Specified secondary outcomes were measures of substance use problems, craving, confidence, mood, pain, and pandemic-related mental health effects. The 15-item Short Inventory of Problems—Alcohol and Drugs (SIP-AD) assessed substance use problems in the past 30 days (Blanchard et al., 2003). The 10-item Drug Abuse Screening Test-10 (DAST-10) (Skinner, 1982) assessed drug-related consequences in the past 2 months, excluding alcohol and tobacco; total scores of 3+ indicate significant drug abuse problems. The DAST-10 item concerning drug-related medical problems was a screening exclusion criterion; hence, the sample’s total possible range was 0−9. Due to an online programming glitch, DAST-10 was not administered at EOT. Craving was assessed as: “In the past 7 days, how much were you bothered by cravings or urges to drink alcohol or use drugs?” from not at all (0) to extremely (4). The Brief Situational Confidence Questionnaire (Breslin et al., 2000) assessed self-confidence to resist the urge “right now” to drink heavily (self-defined) or use drugs in different situations from 0% “not at all confident” to 100% “totally confident.” The 8-item Patient Health Questionnaire (PHQ-8) assessed depressive symptoms (Kroenke et al., 2009). The 7-item Generalized Anxiety Disorder (GAD-7) scale assessed anxiety (Spitzer et al., 2006). Pain was assessed with a rating scale from 0 “no pain” to 100% “worst pain imaginable.” Pain interference in one’s daily life was reported for the past 4 weeks, from not at all (0) to extremely (4). A 6-item measure assessed pandemic-related mental health effects experienced in the past 2 weeks including preoccupation with COVID-19 and worry about self/others’ health (0 = never to 4 = most of the time); stressfulness of changes in social contacts and in one’s lifestyle and worsening of mental/emotional health (0 = not at all to 4 = extremely); and sleep disruption (0 = no change, 1 = sleeping a little more/less, 2 = sleeping a lot more/less). The pandemic-related mental health effects total score had a possible range of 0–22 (Lang, 2020).
Serious adverse events occurring during the 8-week study were assessed including substance use-related hospitalization, suicide attempt, alcohol or drug overdose, and severe withdrawal (e.g., delirium tremens). Positive endorsements were queried for timing, diagnosis, and resolution and reported to the study’s Data Safety Monitoring Board within 72 h of the team learning of the event.
At EOT, participants randomized to W-SUDs completed the Usage Rating Profile-Intervention (URP-I) Feasibility (6 items) and Acceptability (6 items) scales (Briesch et al., 2013), the 8-item Client Satisfaction Questionnaire (CSQ-8) (Larsen et al., 1979), and the Working Alliance Inventory-Short Revised (WAI-SR) with three 4-item subscales assessing development of an affective bond in treatment and level of agreement with treatment goals and treatment tasks (Hatcher and Gillaspy, 2006).
W-SUDs app use metrics included the number of days used; messages sent; modules completed; and mood, craving, and pain ratings submitted. Lesson acceptability was indicated thumbs up or down.
2.5. Data analyses
Paired samples t-tests and chi-square tests compared the groups on baseline variables. EOT survey completion was 84% (152/180) and significantly higher for participants identifying as non-Hispanic white (89%) compared to other racial/ethnic groups (74%) and for participants with a college degree (89%) relative to high school degree (74%) (p-values<.01). Retention also was higher for participants who at baseline reported more pandemic-related mental health effects and lower levels of craving, pain, and pain interference (p-values<.05). Of the 17 treatment participants lost to follow-up, 16 never registered with the W-SUDs app. Fig. 1 shows rates of noncompliance by condition. Two participants randomized to waitlist (2%) downloaded Woebot (not SUD-tailored) from the app store during the study period. Analyses followed the intent-to-treat protocol.
General Linear Models (multivariate models) tested for group differences in changes in the primary outcome and secondary outcomes. The dependent variables were baseline to EOT change scores. The models adjusted for baseline group differences and applied weights to adjust for correlates of study retention. Participant weights were calculated as the inverse of predicted probability values from a logistic regression model predicting EOT retention with race/ethnicity, education, craving, pain intensity, and pandemic-related mental health effects. Pain rating and pain interference were highly correlated (r = .74), due to missing data and multicollinearity, the pain rating item was not included. To examine associations among outcomes, bivariate correlations were run for the changes scores. Lastly, survey data were linked to treatment participants’ app use metrics and tested for associations.
3. Results
3.1. Sample description
The sample (N = 180) averaged 40 years of age (SD = 12), was 65% female, 68% non-Hispanic white, and 56% partnered, residing in 34 U.S. states. At baseline, 3% (n = 6) reported a positive COVID-19 test or diagnosis. At EOT, one additional participant reported COVID-19 diagnosis.
Nearly a third (29%) of the sample identified as essential service workers; 56% of the sample reported pandemic-related employment disruption; 21% had difficulty getting food or medical treatment in the past 2 weeks. Most reported moderate-to-extremely negative pandemic-related effects on their mental/emotional health (64%); lifestyle (62%); and social connections (71%); 59% had disturbed sleep.
Most (77%) identified alcohol as their problematic substance, followed by cannabis (28%), stimulants (12%), and other (14%, e.g., club drugs, opioids, sedatives). A majority (59%) reported a lifetime psychiatric diagnosis, most commonly anxiety disorders (40%) and unipolar depression (28%); 12% reported a SUD diagnosis; 40% reported multiple diagnoses. Few (14%) were currently in therapy; 31% were currently taking psychiatric medication. Engagement in SUD treatment in the past 30 days was reported by 28% of participants, including 7% taking medications to reduce substance use.
The full sample’s mean scores at baseline were 30 (SD = 19) for past-month substance use occasions; 12 (SD = 11) on the SIP-AD; 2 (SD = 2) on the DAST-10; 2 (SD = 1) for craving; 59% (SD = 23%) for confidence to resist substance use urges; and 12 (SD = 5) for pandemic-related mental health effects. Sample baseline mean scores on the PHQ-8 9 (SD = 6), GAD-7 8 (SD = 6), pain rating 32 (SD = 28), and pain interference 1 (SD = 1) were in the mild range. Baseline characteristics by group are shown in Table 1, Table 2 . Waitlist participants had significantly higher CAGE-AID and DAST-10 scores than participants randomized to W-SUDs. All other baseline group comparisons were not statistically significant.
Table 2.
COVID-19 Pandemic-Related Measures by Group at Baseline.
| Variable | W-SUDS Group (N = 88) |
Waitlist Group (N = 92) |
|---|---|---|
| % (n) | % (n) | |
| Employment Status Due to the Pandemic | ||
| Unaffected by the Pandemic | 44.3% (39) | 44.6% (41) |
| Working from Home due to the Pandemic | 27.3% (24) | 25.0% (23) |
| Employment Temporarily or Permanently Stopped | 12.5% (11) | 18.5% (17) |
| Added or Changed Jobs | 4.5% (4) | 6.5% (6) |
| Other / Not Applicable (e.g., retired, homemaker) | 11.4% (10) | 5.4% (5) |
| Designated an Essential Service Worker in the Pandemic | ||
| Yes | 25.0% (22) | 32.6% (30) |
| No/Don’t Know | 75.0% (66) | 67.4% (62) |
| Changes in Income due to the Pandemic | ||
| No Change in Income / Don’t Know | 55.7% (49) | 55.4% (51) |
| Reduced Income | 35.2% (31) | 33.7% (31) |
| Increased Income | 9.1% (8) | 10.9% (10) |
| COVID Precautions Past 2 Weeks (% most of time)NA | ||
| Staying 6 ft. away from other People (n=179) | 71.6% (63) | 71.4% (65) |
| Staying/working at home (n=153) | 58.9% (43) | 57.5% (46) |
| Avoiding large social gatherings (n=174) | 83.5% (71) | 80.9% (72) |
| Avoiding small social gatherings (n=175) | 40.7% (35) | 61.8% (55) |
| Avoiding any non-essential travel (n=170) | 56.6% (47) | 56.3% (49) |
| Avoiding bars (n=163) | 79.5% (62) | 76.5% (65) |
| Difficulty getting Necessities (food, medication, medical help) in the Past 2 Weeks due to the Pandemic | 18.2% (16) | 23.9% (22) |
| Pandemic-related Mental Health Effects | ||
| Preoccupation with the COVID-19 Virus (often/most of time) | 62.5% (55) | 50.0% (46) |
| Worrying about Health of Self & Others (often/most of time) | 52.3% (46) | 59.8% (55) |
| Stressfulness of Changes in Social Contacts (mod-to-extreme) | 71.6% (63) | 69.6% (64) |
| Stressfulness of Changes in Lifestyle (mod-to-extreme) | 62.5% (55) | 62.0% (57) |
| Worsening of Mental/Emotional Health (mod-to-extreme) | 64.5% (57) | 63.0% (58) |
| Pandemic Effects on Sleep | ||
| Sleeping a Lot More | 9.1% (8) | 6.5% (6) |
| Sleeping a Little More | 9.1% (8) | 8.7% (8) |
| No Change in Sleep | 31.8% (28) | 33.7% (31) |
| Sleeping a Little Less | 34.1% (30) | 30.4% (28) |
| Sleeping a Lot Less | 15.9% (14) |
20.7% (19) |
| Mean (SD) |
Mean (SD) |
|
| Pandemic-related Mental Health Effects Score | 12.1 (4.3) | 12.3 (5.1) |
3.2. Changes in substance use and mental health outcomes by group
Table 3 summarizes the results of eight general linear models testing group differences in the primary outcome and secondary outcomes. From baseline to EOT, participants randomized to W-SUDs reduced their substance use occasions significantly more than the waitlist group (F(1,148) = 4.53, p = .035). The estimated marginal mean reductions in past 30-day substance use occasions were -9.6 (SE = 2.3) for W-SUDs versus -3.9 (SE = 2.2) for the waitlist control. Confidence gains were on average two-fold greater for W-SUDs than the waitlist group, but not statistically significant (p = .175). Substance use problems, craving, depressive symptoms, anxiety, pain ratings, and pandemic-related mental health effects declined over time with no statistically significant difference between groups.
Table 3.
Tests of Group Differences in Baseline to End-of-Treatment (EOT) Change Scores (N = 152).
| W-SUDS (N = 71) | Waitlist (N = 81) | ||||
|---|---|---|---|---|---|
| Dependent Variable | Estimated Marginal Mean (SE) | Estimated Marginal Mean (SE) | F | p | Eta2 |
| Primary Outcome | |||||
| Substance Use Occasions | −9.1 (2.0) | −3.3 (1.8) | 4.35 | .039 | .029 |
| Secondary Outcomes | |||||
| Short Inventory of Problems – Alcohol and Drugs (SIP-AD) | −5.3 (1.1) | −4.7 (1.0) | 0.15 | .701 | .001 |
| Confidence Score (0−100%) | 11.8 (3.4) | 5.3 (3.1) | 1.86 | .175 | .012 |
| Cravings Past 7-days | −0.5 (0.1) | −0.4 (0.1) | 0.04 | .843 | .000 |
| Depressive Symptoms (PHQ-8) | −1.6 (0.6) | −1.7 (0.6) | 0.03 | .854 | .000 |
| Anxiety Symptoms (GAD-7) | −1.6 (0.6) | −0.7 (0.5) | 1.19 | .278 | .008 |
| Pandemic-related MH Effects | −1.8 (0.5) | −1.4 (0.5) | 0.30 | .586 | .002 |
| Pain Rating§ (0−100%) | −5.1 (2.8) | −0.8 (2.5) | 1.23 | .269 | .008 |
Note: All dependent variables are changes scores calculated as EOT – baseline. General Linear Models were run to test for group differences adjusting for baseline differences (CAGE-AID and DAST-10) and applying weighted least squares analyses to adjust for differential retention. Weights were calculated as the inverse of predicted probability values from a logistic regression model of retention at EOT. The logistic regression model included baseline measures univariately associated with retention: non-Hispanic White race/ethnicity, college degree, craving, pain interference, and pandemic mental health effects. Eta2 = 0.01 indicates a small effect; Eta2 = 0.06 indicates a medium effect; Eta2 = 0.14 indicates a large effect. §Missing a baseline pain rating from one participant in the waitlist condition (n = 80). MH = mental health.
At EOT, 24% (37/152) of participants reported engaging in SUD treatment (other than W-SUDs) during the 2-month study period with no significant difference by study condition (X2 (1,152) = 0.52, p = .572). Engagement in SUD treatment during the study period, added as a covariate to the model of substance use occasions, was not significant (F(1,147) = 0.31, p = .577) and did not alter the treatment effect for W-SUDs (F(1,147) = 4.53, p = .035). In the models of secondary outcomes, SUD treatment engagement was significantly associated with only anxiety. Engaging in SUD treatment during the study period was associated with less reductions in anxiety (p < .001).
Reduction in substance use occasions was significantly associated with reduction in substance use problems and both measures were significantly associated with gains in confidence and decreased cravings, depressive and anxiety symptoms, and pandemic-related mental health effects (Table 4 ). Confidence gains were significantly associated with improved mood and reduced cravings. Reductions in mood symptoms and pandemic-related mental health effects were significantly correlated.
Table 4.
Correlations among change scores (baseline to end-of-treatment) for substance use occasions, substance use problems, confidence, depressive symptoms, and anxiety (N = 152).
| SIP-AD | Confidence | Craving | PHQ-8 | GAD-7 | Pandemic MH effects | |
|---|---|---|---|---|---|---|
| Substance Use Occasions | .34*** | −.29*** | .35*** | .26** | .23** | .20* |
| Substance Use Problems, SIP-AD | 1 | −.39*** | .55*** | .50*** | .49*** | .33** |
| Confidence Score | 1 | −.51*** | −.41*** | −.43*** | −.16 | |
| Craving Past 7-days | 1 | .35*** | .41*** | .24** | ||
| Depressive Symptoms, PHQ-8 | 1 | .65*** | .33*** | |||
| Anxiety Symptoms, GAD-7 | 1 | .25** | ||||
| Pandemic-related MH Effects | 1 |
*p < .05, **p < .01, ***p < .001. MH = mental health.
3.3. Serious adverse events
Of the 152 participants completing the EOT assessment, one reported a serious adverse event. The individual, randomized to waitlist, was hospitalized for alcohol detoxification. The DSMB deemed the event to be unrelated to study involvement.
3.4. W-SUDs use and acceptability
Table 5 summarizes treatment participants’ use of the W-SUDs app. Of the 88 participants randomized to W-SUDs, 77 (88%) registered with the app, using the app an average of 34.9 days (SD = 28.9, Median = 31), and sending on average 1052 in-app text messages (SD = 878, Median = 881). Including the 11 participants who did not register with W-SUDs in the denominator, the treatment group averaged 920 in-app text messages (SD = 892, Median = 701). Engagement ranged from 77 participants sending on average 172 messages (SD = 105, Median = 182) at week 1–50 participants averaging 126 messages (SD = 89, Median = 111) at week 8; 52 participants (68% of the 77 who registered) continued to message W-SUDs after 8 weeks. The correlation between the total count of in-app text messages and reduction in self-reported substance use occasions from baseline to EOT was r = −.23, p = .06, n = 68.
Table 5.
W-SUD App Usage among Treatment Participants (N = 88).
| N (%) of participants | Descriptive Statistics among those Engaged |
|||
|---|---|---|---|---|
| engaged | Mean (SD) | Median | IQR | |
| Days Used | 77 (88%) | 34.9 (28.9) | 31.0 | 12.0, 50.5 |
| In-App Text Messages | 77 (88%) | 1052.0 (878.0) | 881 | 351.5, 1626.5 |
| Completed Modules | 74 (84%) | 20.0 (12.0) | 17.5 | 9.8, 31.0 |
| Submitted Ratings | ||||
| Mood | 70 (80%) | 35.2 (27.6) | 32.0 | 14.8, 54.3 |
| Craving | 70 (80%) | 6.3 (2.8) | 7.5 | 4.0, 9.0 |
| Pain | 68 (77%) | 6.3 (2.7) | 7.5 | 5.0, 9.0 |
Most treatment participants (77%–84%) completed modules and submitted mood, craving, and pain ratings; 96% of completed lessons received a thumbs-up rating. The most frequently completed lesson, “COVID-19 Perspective,” received thumbs-up from 61 of 63 treatment participants (97%). Highest-rated lessons (100% thumbs up), completed by a majority of treatment participants, were on: urge surfing, labeling, the function of emotions, and gratitude. Participants provided craving and pain ratings nearly weekly, and mood ratings about four times a week. App engagement metrics were strongly correlated, ranging from r = .66 for the volume of mood ratings and in-app text messages to r = .98 for entry of craving and pain ratings (p-values<.001).
Participants currently in therapy rated lessons lower on average (M = 85% thumbs up) compared to those formerly or never in therapy (both M = 98% thumbs up, F(2,61) = 4.09, p = .022). Men (M = 48) completed more mood ratings than women (M = 33, F(1,63) = 4.02, p = .049). No other significant differences were found in app usage by participant characteristics.
Table 6 summarizes treatment participants’ (n = 66) acceptability ratings of W-SUDs at EOT. Pairwise t-test comparisons for the three WAI-SR subscales, a measure of therapeutic alliance, indicated significantly higher ratings of W-SUDs on affective bond formation relative to agreement on treatment tasks (t(65) = 3.96, p < .001) and goals (t(65) = 2.91, p = .005); scores for agreement on treatment goals and tasks did not significantly differ (t(65) = −1.78, p = .081). On the CSQ-8 individual items, most indicated that W-SUDs provided the kind of service they wanted (86%); provided good-to-excellent quality of interaction (89%); helped them deal more effectively with their problems (89%); and met most/almost all of their needs (74%). Additionally, 80% were mostly/very satisfied with the amount of help received and 85% were mostly/very satisfied with W-SUDs overall; 83% would return to W-SUDs; and 88% would recommend W-SUDs to a friend.
Table 6.
Treatment Participants’ W-SUDs Feasibility & Acceptability Ratings (N = 66).
| Possible Range | Mean (SD) | |
|---|---|---|
| Usage Rating Profile-Intervention (URP-I) | ||
| Acceptability (6 items) | 6−36 | 29.0 (4.8) |
| Feasibility (6 items) | 6−36 | 32.0 (3.6) |
| Client Satisfaction Questionnaire (CSQ-8, 8 items) | 8−32 | 25.5 (5.0) |
| Working Alliance Inventory-Short Revised (WAI-SR) | 12−60 | 44.0 (12.5) |
| Agreement with treatment goals (4 items) | 4−20 | 14.4 (4.7) |
| Agreement with treatment tasks (4 items) | 4−20 | 13.9 (4.5) |
| Affective bond formation in treatment (4 items) | 4−20 | 15.7 (4.6) |
Some significant differences were found in W-SUDs acceptability by participant baseline characteristics. Non-Hispanic white and ever-married participants gave higher URP-I feasibility ratings; participants with less education, no prior therapy experience, and greater substance use occasions had higher CSQ-8 satisfaction scores (p’s<.05). Baseline to EOT reductions in substance use occasions, depression, and anxiety significantly correlated with W-SUDs acceptability (r’s = −.26 to −.39, p < .05). In-app thumbs up ratings correlated with CSQ-8 (r = .52, p < .001) and WAI-SR (r = .44, p < .001). URP-I feasibility correlated with days of W-SUDs app use (r = .27, p = .030) and the number of in-app text messages (r = .30, p = .015), mood ratings (r = .34, p = .007) and completed modules (r = .25, p = .048).
4. Discussion
In a randomized controlled trial conducted in summer 2020, during the COVID-19 pandemic, W-SUDs, a therapeutic relational agent, had decreased substance use occasions relative to a waitlist control group. Participants who reduced their substance use occasions from baseline to EOT reported increases in confidence to resist urges and fewer substance use problems, cravings, depressive and anxiety symptoms, and pandemic-related mental health effects. Hence, successfully reducing substance use, even if not achieving complete abstinence, was associated with fewer negative consequences and mental health concerns.
Improvements observed in secondary substance use and mood outcomes did not differ by group. Prior studies of W-SUDs and Woebot have found significant treatment effects on mood (Fitzpatrick et al., 2017; Prochaska et al., 2021). Failure to replicate may relate to the sample’s lower depression and anxiety scores at baseline or may relate to pandemic-related transitions such as gradual reopening of services. A novel moment in time, pandemic-related secular trends are possibly anomalous and unpredictable making it challenging to generalize findings. Though changes in secondary outcomes did not differ significantly by group, changes in secondary outcomes were significantly associated with improvements in the primary outcome of substance use occasions.
The W-SUDs app registration rate was 88%, better than our initial study (Prochaska et al., 2021) and comparable or better than other successful mobile health interventions (Cliffe et al., 2020). More participants used W-SUDs early in treatment; however, app activity remained high through 8 weeks and for many participants extended past 8-weeks. W-SUDs improvements since our initial pilot include content refinement based on previous W-SUDs’ participant feedback and integration of a refined app-onboarding process. Compared to our pilot, app use metrics in the current study nearly doubled and acceptability ratings increased on all measures. As found in our previous study, W-SUDs scored highest on the WAI-SR on affective bond formation, which is notable given that Woebot is expressly non-human. That the sum count of total in-app texts was weakly correlated with reduction in substance use occasions is consistent with acknowledgement in the digital health space of the need for promoting “effective engagement” rather than simply more engagement to achieve intended behavior change outcomes (Yardley et al., 2016). In the current study, reduction in substance use occasions was associated with W-SUDs treatment acceptability, feasibility, and satisfaction.
Engagement in SUD treatment services during the study period did not predict reductions in substance use occasions; affect the strength of the treatment effect of W-SUDs; or correlate with measures of W-SUDs use, acceptability, feasibility, or satisfaction. W-SUDs appears useful and appropriate for adults with substance use concerns across the severity continuum.
The current findings support the utility of AI applications for reflecting tailored empathy, at scale, and support and extend a growing body of literature on the use of conversational agents (or chatbots) to support behavioral health. A systematic review found that most online interventions targeting problematic substance use produced significant short-term improvement on at least one measure of problematic substance use (Giroux et al., 2017). The current findings also contribute to research on digital health interventions for delivering mental health and substance use treatment to communities affected by natural disasters or other widespread stressors (Ruggiero et al., 2006, 2012; Strudwick et al., 2021).
Prior studies of digital therapeutics, including programs addressing substance misuse, have indicated lower participation among people without college degrees (Potdar et al., 2020; Ruggiero et al., 2006). Our finding of higher acceptability of W-SUDs among those with less education provides reassurance to concerns that digital therapeutics may increase health inequities (Azzopardi-Muscat and Sørensen, 2019) and reinforces the literature demonstrating that relational agents are approachable and usable for people with lower reading literacy (Bickmore and Gruber, 2010). In-hand digital therapeutics such as W-SUDs have utility for serving populations restricted in time, transportation, and financial resources for attending in-person treatment. Further, use of W-SUDs proved feasible and efficacious during a pandemic that called for physical distancing. Broader applications for reaching under-served populations post-pandemic are anticipated.
Study strengths include the randomized design and low study dropout (i.e., missingness of outcome data). Analyses adjusted for baseline differences between conditions and predictors of attrition. Noncompliance, or crossover between conditions, was 11% overall; analyses followed randomization (i.e., intent-to-treat). The current study was limited to short-term outcomes, and the sample was predominately female, identifying as non-Hispanic white, and employed full- or part-time pre-pandemic. Future research on W-SUDs will use a randomized design with an active comparator, with longer follow-up, and recruitment of a more diverse population using quotas to ensure racial/ethnic diversity in sampling. All data for this study were collected remotely and self-reported. Self-reported substance use can be subject to recall and social desirability bias; however, being virtual and not abstinence-only focused, the demand characteristics in the current study were low. The pre/post controlled design also mitigates concern about the influence of bias in reporting. The outcomes were standard self-report measures with demonstrated validity and reliability. Few participants were misusing opioids, likely due to study exclusion designed to mitigate risk (i.e., engagement with medication-assisted treatment, no history of opioid overdose requiring naloxone). Notably, over 1100 people with interest in a program for those with substance use concerns were excluded due to low severity on the CAGE-AID screener. Worth testing is the utility of digital health programs for early intervention on the misuse of substances that is subsyndromal.
In a randomized controlled trial evaluation, W-SUDs, a fully automated therapeutic relational agent was efficacious in reducing substance use occasions, had high acceptability, and was feasible to access daily, even in the context (and sometimes confines) of a global pandemic. The current findings provide support for a digital therapeutic such as W-SUDs for addressing substance misuse.
Funding
Conduct of the study was supported by the National Institute on Drug Abuse (NIDA) with a Small Business Innovations Research Award #R44-D048712 and a COVID-19 related supplement, and by the National Heart, Lung, and Blood Institute (NHLBI) #T32HL007034. The article’s contents are solely the responsibility of the authors and do not necessarily represent the official views of NIDA and NHLBI.
Author contributions
JJP, AR and AD designed the study and acquired funding. JJP and AR supervised the study recruitment and implementation and coordinated the research activities. EAV and SP provided intellectual input to the study assessments. KRW aided in intervention content development. AC led institutional approvals. AC and SP led study recruitment, data accrual, and data management. JJP, AC, EAV, DDM and MB had access to the study data downloaded from Qualtrics. MB created the randomization and advised on data analyses. JJP, EAV and DDM performed the data analyses. JJP drafted the manuscript and incorporated feedback from all coauthors.
Trial registration
ClinicalTrials.Gov NCT04096001; http://clinicaltrials.gov/ct2/show/NCT04460027.
Declaration of Competing Interest
Sarah Pajarito, MA, Ken Weingardt, PhD, Alison Darcy, PhD, and Athena Robinson, PhD are Woebot Health employees. Other authors: None.
Acknowledgments
We acknowledge the study’s Data Safety Monitoring Members Travis Cook, PhD, Julie Hamlin, LCSW, Danielle Ramo, PhD, and Kelly Young-Wolff, PhD, MPH for their oversite of the study procedures and outcomes. We are grateful for the ongoing guidance from Leonardo Angelone, PhD and Victor Prikhodko, MBA at NIDA.
References
- Azzopardi-Muscat N., Sørensen K. Towards an equitable digital public health era: promoting equity through a health literacy perspective. Eur. J. Public Health. 2019;29(Supplement_3):13–17. doi: 10.1093/eurpub/ckz166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baumel A., Muench F., Edan S., Kane J.M. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J. Med. Internet Res. 2019;21(9) doi: 10.2196/14567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berry K., Salter A., Morris R., James S., Bucci S. Assessing therapeutic alliance in the context of mHealth interventions for mental health problems: development of the Mobile Agnew Relationship Measure (mARM) questionnaire. J. Med. Internet Res. 2018;20(4):e90. doi: 10.2196/jmir.8252. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickmore T., Gruber A. Relational agents in clinical psychiatry. Harv. Rev. Psychiatry. 2010;18(2):119–130. doi: 10.3109/10673221003707538. [DOI] [PubMed] [Google Scholar]
- Blanchard K.A., Morgenstern J., Morgan T.J., Labouvie E.W., Bux D.A. Assessing consequences of substance use: psychometric properties of the Inventory of Drug Use Consequences. Psychol. Addict. Behav. 2003;17(4):328–331. doi: 10.1037/0893-164X.17.4.328. [DOI] [PubMed] [Google Scholar]
- Boumparis N., Schulte M.H.J., Riper H. Digital mental health for alcohol and substance use disorders. Curr. Treat. Options Psychiatry. 2019;6(4):352–366. [Google Scholar]
- Breslin F.C., Sobell L.C., Sobell M.B., Agrawal S. A comparison of a brief and long version of the Situational Confidence Questionnaire. Behav. Res. Ther. 2000;38(12):1211–1220. doi: 10.1016/s0005-7967(99)00152-7. [DOI] [PubMed] [Google Scholar]
- Briesch A.M., Chafouleas S.M., Neugebauer S.R., Riley-Tillman T.C. Assessing influences on intervention implementation: revision of the Usage Rating Profile-Intervention. J. Sch. Psychol. 2013;51(1):81–96. doi: 10.1016/j.jsp.2012.08.006. [DOI] [PubMed] [Google Scholar]
- Brorson H.H., Ajo Arnevik E., Rand-Hendriksen K., Duckert F. Drop-out from addiction treatment: a systematic review of risk factors. Clin. Psychol. Rev. 2013;33(8):1010–1024. doi: 10.1016/j.cpr.2013.07.007. [DOI] [PubMed] [Google Scholar]
- Brown R.L., Rounds L.A. Conjoint screening questionnaires for alcohol and other drug abuse: criterion validity in a primary care practice. Wis. Med. J. 1995;94(3):135–140. [PubMed] [Google Scholar]
- Clay J.M., Parker M.O. Alcohol use and misuse during the COVID-19 pandemic: a potential public health crisis? Lancet Public Health. 2020;5(5):e259. doi: 10.1016/S2468-2667(20)30088-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cliffe B., Croker A., Denne M., Smith J., Stallard P. Digital cognitive behavioral therapy for insomnia for adolescents with mental health problems: feasibility open trial. JMIR Ment. Health. 2020;7(3) doi: 10.2196/14842. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cook J.E., Doyle C. Working alliance in online therapy as compared to face-to-face therapy: preliminary results. Cyberpsychology Behav. 2002;5(2):95–105. doi: 10.1089/109493102753770480. [DOI] [PubMed] [Google Scholar]
- Czeisler M.É., Lane R.I., Petrosky E., Wiley J.F., Christensen A., Njai R., Weaver M.D., Robbins R., Facer-Childs E.R., Barger L.K., Czeisler C.A., Howard M.E., Rajaratnam S.M.W. Mental health, substance use, and suicidal ideation during the COVID-19 pandemic— united States, June 24-30, 2020. MMWR. 2020;69(32):1049–1057. doi: 10.15585/mmwr.mm6932a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fitzpatrick K.K., Darcy A., Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Ment. Health. 2017;4(2):e19. doi: 10.2196/mental.7785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Giroux I., Goulet A., Mercier J., Jacques C., Bouchard S. Online and mobile interventions for problem gambling, alcohol, and drugs: a systematic review. Front. Psychol. 2017;8(954) doi: 10.3389/fpsyg.2017.00954. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hatcher R.L., Gillaspy J.A. Development and validation of a revised short version of the working alliance inventory. Psychother. Res. 2006;16(1):12–25. [Google Scholar]
- Kroenke K., Strine T.W., Spitzer R.L., Williams J.B.W., Berry J.T., Mokdad A.H. The PHQ-8 as a measure of current depression in the general population. J. Affect. Disord. 2009;114(1):163–173. doi: 10.1016/j.jad.2008.06.026. [DOI] [PubMed] [Google Scholar]
- Lang A.J. 2020. Complementary and Integrative Research (CAIR) Pandemic Impact Questionnaire (C-PIQ) San Diego, CA. [Google Scholar]
- Larsen D.L., Attkisson C.C., Hargreaves W.A., Nguyen T.D. Assessment of client/patient satisfaction: development of a general scale. Eval. Program Plann. 1979;2(3):197–207. doi: 10.1016/0149-7189(79)90094-6. [DOI] [PubMed] [Google Scholar]
- Lavoie K., Bacon S. Montreal Behavioural Medicine Centre; 2020. International COVID-19 Awareness and Responses Evaluation (iCARE) Study. [Google Scholar]
- Lennox R., Dennis M.L., Scott C.K., Funk R. Combining psychometric and biometric measures of substance use. Drug Alcohol Depend. 2006;83(2):95–103. doi: 10.1016/j.drugalcdep.2005.10.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lucas G.M., Gratch J., King A., Morency L.-P. It’s only a computer: virtual humans increase willingness to disclose. Comput. Human Behav. 2014;37:94–100. [Google Scholar]
- Moss H.B., Chen C.M., Yi H.Y. Measures of substance consumption among substance users, DSM-IV abusers, and those with DSM-IV dependence disorders in a nationally representative sample. J. Stud. Alcohol Drugs. 2012;73(5):820–828. doi: 10.15288/jsad.2012.73.820. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Potdar R., Thomas A., DiMeglio M., Mohiuddin K., Djibo D.A., Laudanski K., Dourado C.M., Leighton J.C., Ford J.G. Access to internet, smartphone usage, and acceptability of mobile health technology among cancer patients. Support. Care Cancer. 2020;28(11):5455–5461. doi: 10.1007/s00520-020-05393-1. [DOI] [PubMed] [Google Scholar]
- Prochaska J.J., Vogel E.A., Chieng A., Kendra M., Baiocchi M., Pajarito S., Robinson A. A therapeutic relational agent for reducing problematic substance use (Woebot): development and usability study. J. Med. Internet Res. 2021;23(3) doi: 10.2196/24850. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ruggiero K.J., Resnick H.S., Acierno R., Coffey S.F., Carpenter M.J., Ruscio A.M., Stephens R.S., Kilpatrick D.G., Stasiewicz P.R., Roffman R.A., Bucuvalas M., Galea S. Internet-based intervention for mental health and substance use problems in disaster-affected populations: a pilot feasibility study. Behav. Ther. 2006;37(2):190–205. doi: 10.1016/j.beth.2005.12.001. [DOI] [PubMed] [Google Scholar]
- Ruggiero K.J., Resnick H.S., Paul L.A., Gros K., McCauley J.L., Acierno R., Morgan M., Galea S. Randomized controlled trial of an internet-based intervention using random-digit-dial recruitment: the Disaster Recovery Web project. Contemp. Clin. Trials. 2012;33(1):237–246. doi: 10.1016/j.cct.2011.10.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Skinner H.A. The drug abuse screening test. Addict. Behav. 1982;7(4):363–371. doi: 10.1016/0306-4603(82)90005-3. [DOI] [PubMed] [Google Scholar]
- Spitzer R.L., Kroenke K., Williams J.B.W., Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 2006;166(10):1092–1097. doi: 10.1001/archinte.166.10.1092. [DOI] [PubMed] [Google Scholar]
- Strudwick G., Sockalingam S., Kassam I., Sequeira L., Bonato S., Youssef A., Mehta R., Green N., Agic B., Soklaridis S., Impey D., Wiljer D., Crawford A. Digital interventions to support population mental health in Canada during the COVID-19 pandemic: rapid review. JMIR Ment. Health. 2021;8(3) doi: 10.2196/26550. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Substance Abuse and Mental Health Services Administration . Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; Rockville, MD: 2019. Key Substance Use and Mental Health Indicators in the United States: Results from the 2018 National Survey on Drug Use and Health. [Google Scholar]
- The Nielsen Company . 2020. Rebalancing the ‘COVID-19 Effect’ on Alcohol Sales.https://www.nielsen.com/us/en/insights/article/2020/rebalancing-the-covid-19-effect-on-alcohol-sales/ [Google Scholar]
- Tielman M.L., Neerincx M.A., van Meggelen M., Franken I., Brinkman W.P. How should a virtual agent present psychoeducation? Influence of verbal and textual presentation on adherence. Technol. Health Care. 2017;25(6):1081–1096. doi: 10.3233/THC-170899. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Torous J., Nicholas J., Larsen M.E., Firth J., Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid. Ment. Health. 2018;21(3):116–119. doi: 10.1136/eb-2018-102891. [DOI] [PMC free article] [PubMed] [Google Scholar]
- U.S. Department of Health and Human Services . HHS; Washington, DC: 2016. Facing Addiction in America: The Surgeon General’s Report on Alcohol, Drugs, and Health. [PubMed] [Google Scholar]
- Vaidyam A.N., Wisniewski H., Halamka J.D., Kashavan M.S., Torous J.B. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can. J. Psychiatry. 2019;64(7):456–464. doi: 10.1177/0706743719828977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vangst, LeafLink, Flowhub . 2020. The State of the Cannabis Industry 2020.https://www.stateofthecannabisindustry.com/ (Accessed 4/8/2021) [Google Scholar]
- Yardley L., Spring B.J., Riper H., Morrison L.G., Crane D.H., Curtis K., Merchant G.C., Naughton F., Blandford A. Understanding and promoting effective engagement with digital behavior change interventions. Am. J. Prev. Med. 2016;51(5):833–842. doi: 10.1016/j.amepre.2016.06.015. [DOI] [PubMed] [Google Scholar]