A randomized clinical trial of smartphone self-managed recovery support Services

Abstract

This study examines the effectiveness of smartphone-based ecological momentary interventions (EMI) and assessments (EMA), delivered separately and combined, to provide recovery support following substance use disorder (SUD) treatment engagement. We recruited adults (N=401) from SUD treatment programs in Chicago and, after engagement for at least two sessions, nights, or medication dosages, we randomly assigned them to one of four conditions that lasted 6 months: (1) EMI only, (2) EMA only, (3) both EMI and EMA, and (4) control condition of neither EMI nor EMA. EMIs provided support for recovery through applications on the phone or links to other resources; EMAs were delivered randomly 5 times per day asking participants to indicate recent substance use and situational risk and protective factors. The primary dependent variable was days of abstinence in the 6 months following study intake. Rates of EMI and EMA utilization indicated high compliance, although EMI use decreased over time. There was a small direct effect of time across conditions (F_(2,734)=4.33,p=0.014, Cohen’s f=0.11) and a small direct effect of time-by-EMI use (F_(2,734)=4.85, p=.009, f=0.11) on days of abstinence. There was no significant direct effect of time-by-EMAs nor interaction effect of time-by-EMI-by-EMA. However, secondary path model analyses showed a small but significant indirect effect of EMA on abstinence via EMI use. Stepwise modeling identified a simplified model based on the proportion of weeks using ≥ 1 EMI and the EMI to listen to music, which predicted 7.2% of the variance in days of abstinence (F_(2,195,)=7.56, p<.001). Combined delivery of EMI and EMA shows potential for increasing abstinence above and beyond the effect of SUD treatment engagement and for addressing the limited national capacity for recovery support.

1. Introduction

With the understanding that substance use disorders (SUD) are often chronic and cyclical in nature (Dennis, Scott, & Funk, 2003; Dennis & Scott, 2012; Scott & Dennis, 2009; Scott, Dennis, & Foss, 2005; Scott, Dennis, & Lurigio, 2017), the SUD treatment field has increasingly emphasized interventions that provide ongoing recovery support to patients following SUD treatment discharge. Such interventions have focused on teaching self-management skills to address risks for relapse, providing recovery support services—such as recovery coaching and facilitating mutual-help group participation (Gonzales-Castaneda et al., 2019)—and providing support through continuing care interventions (Chi, Parthasarathy, Mertens, & Weisner, 2011; McKay & Hiller-Sturmhofel, 2011). Continuing care with monitoring has been associated with better outcomes for multiple chronic conditions in general (McLellan, Lewis, O’Brien, & Kleber, 2000) and specifically for SUDs (Dennis et al., 2003; Dennis & Scott, 2012; Scott et al., 2017). However, SUD patients rarely receive it (McLellan et al., 2005; White, Boyle, & Loveland, 2002). Another common goal of long-term chronic disease management is the provision of self-managed interventions and monitoring that are becoming increasingly assisted with technology (Chodosh et al., 2005; Swendeman, Ingram, & Rotheram-Borus, 2009; WHO, 2016).

Smartphone apps (aka mobile or mHealth interventions) have been used to provide support for self-management of chronic disorders, such as diabetes, hypertension, and asthma (de Jongh, Gurol-Urganci, Vodopivec-Jamsek, Car, & Atun, 2012) and to promote positive health behaviors generally. They are a promising option to expand the capacity of care/disease management outside of traditional treatment settings (Byambasuren, Sanders, Beller, & Glasziou, 2018; Han & Lee, 2018; Lindhiem, Bennett, Rosen, & Silk, 2015; Zhao, Freeman, & Li, 2016; WHO, 2016). Systematic reviews have reported positive findings of smartphone interventions to deliver relapse prevention prompts, recovery support, or contingency management interventions for SUDs (Fowler, Holt, & Joshi, 2016; Gao, Cao, Guo, & Xiao, 2018; Getty, Morande, Lynskey, Weaver, & Metrebian, 2019; Jones et al., 2019; Tofighi, Chemi, Ruiz-Valcarcel, Hein, & Hu, 2019). Yet this literature is limited, with most systematic reviews examining the same two dozen studies that are primarily pilot or exploratory in focus, and only a handful of these studies have directly assessed patient outcomes. Moreover, most studies that have examined patient outcomes used smartphone interventions that were clinician-managed, rather than self-managed. The current study addresses the need for rigorous investigation of smartphone interventions for self-management of SUD recovery support services.

1.1. Intervention model

1.1.1. Ecological Momentary Interventions (EMI)

The self-management of a recovery support intervention used here builds upon the Addiction Comprehensive Health Education Support System (A-CHESS), which is a suite of EMIs (i.e., interventions delivered to patients in everyday settings as they go about their regular daily routines) delivered via mobile apps. Grounded in self-determination theory (Deci & Ryan, 2012), A-CHESS is one of the few apps that has been tested in a large clinical trial. It demonstrated effects of fewer risky drinking days in the past month compared with patients in the control condition (1.39 vs. 2.75 days, p<.01), and significantly increased the probability of abstinence from alcohol for 30 days or more (78.7% vs. 65.5%, p=<.05 (Gustafson et al., 2014). Two key limits of this study were that only 57.6% of the participants in the experimental condition used A-CHESS at least weekly in the last month of the study, and it did not demónstrate effects on days of abstinence from other drugs alone or in combination with alcohol. The framework and elements of A-CHESS formed the basis for the EMI used for this study, with changes and multiple additions to the intervention, as noted in the Methods section.

1.1.2. Ecological Momentary Assessments (EMA)

While no prior experimental studies have demonstrated long-term, post-treatment use of EMAs (i.e., regular monitoring of behaviors and/or experiences in real time in the subject’s natural environment) leads to changes in substance use, self-monitoring is considered a key component of self-management for chronic diseases in general (Bodenheimer, Lorig, Holman, & Grumbach, 2002; Hamine, Gerth-Guyette, Faulx, Green, & Ginsburg, 2015; Hanlon et al., 2017; Swendeman et al., 2015; Whitehead & Seaton, 2016; Wichers et al., 2011). Mindfulness and relapse prevention interventions for SUD that utilize self-monitoring also have demonstrated beneficial effects (Elkins-Brown, Teper, Inzlicht, & Elkins-Brown, 2017; Freedman, Lester, McNamara, Milby, & Schumacher, 2006; Kauer et al., 2012; Marlatt & Donovan, 2005; Witkiewitz & Kirouadc, 2015; Witkiewitz & Marlatt, 2004; Witkiewitz & Marlatt, 2011). Research has employed behavioral self-monitoring for several decades, both as a means of studying behavior and as a component of therapeutic interventions that promote self-management of chronic disorders, such as depression, HIV, tobacco addiction and other SUDs (Carter, Day, Cinciripini, & Wetter, 2007; Wichers et al., 2011). Behavioral record-keeping itself generally increases one’s awareness of the frequency, patterns, and circumstances associated with a target behavior and tends to increase desirable behaviors and decrease negative ones (Kazdin, 1974, 1980; Nelson, 1977). Kauer and colleagues (2012) demonstrated that increasing the rate of monitoring from 1 to 6 times each day was related to increased emotional self-awareness and subsequent reductions in depressive symptoms. In the current study we have introduced EMA to help people self-monitor behaviors associated with risk of relapse to substance use at the time and in the context in which they occur.

1.1.3. Potential interaction of EMI and EMA

Mobile app-based EMIs studied to date (including A-CHESS) require patients to self-initiate use. Self-initiation may be difficult for individuals currently experiencing a crisis or cravings, and it requires the individual to be proactive in a manner uncharacteristic of those with SUD (Bechara, 2003; Koffarnus & Kaplan, 2018). The current study is one of the first to examine the combined effects of EMI and EMA relative to their singular effects. We do not know whether their combination will interact (i.e., the effect of EMA+EMI is greater than expected from the two main effects alone). Even if there is no demonstrated interaction effect, consistent with studies demonstrating positive effects of self-monitoring, we theorize delivery of 5 EMAs at random times each day may also serve as a trigger to increase the amount and consistency of EMI use, thereby indirectly impacting days of abstinence, even in the absence of a direct EMA effect.

1.2. Aims and hypotheses

The primary aim of this trial was to examine the effects of smartphone-based self-managed recovery support interventions via EMIs, EMAs, or their combination on days of abstinence from drugs and alcohol over the course of 6 months following treatment. Our primary hypotheses were that post-treatment: H1a) participants who utilize EMIs will have a greater number of days abstinent from alcohol and drugs (AOD) relative to controls; H1b) participants assigned to receive EMAs will have a greater number of days abstinent from AOD relative to controls; and H1c) there will be an interaction such that the effect of receiving both EMIs and EMAs will be greater than the main effects of each alone. Regardless of whether there is a significant interaction in H1c, we also examine the exploratory hypotheses that: H2a) the pattern of EMI utilization will predict more days of abstinence, and H2b) there will be an indirect effect of EMAs (via increased EMI utilization) on days of AOD abstinence.

2. Methods

2.1. Data source and design

We recruited participants after SUD treatment engagement (National Committee for Quality Assurance [NCQA], 2020). This trial used a 2×2 factorial design with a project manager assigning participants randomly with urn randomization, using G-Rand version 1.1 and equal probabilities to one of the four conditions: EMIs only, EMAs only, combined EMIs + EMAs, or control. At the start of the training, participants in all four groups received a refresher course in relapse prevention. Participants in the three EMI and/or EMA experimental groups received a study-controlled Samsung Galaxy® S5, S6, or S7 smartphone and 6-month data plan, as well as training in its use and the use of the respective apps, and quality assurance monitoring (discussed further). Post-assignment, all four groups still had access to treatment and recovery support services available in the community. We assessed all four groups with interviews and urine tests at enrollment, and 3- and 6-months post-enrollment. We conducted assessments primarily in person at enrollment and at 3- and 6-months post-enrollment, although we conducted a small number of post-enrollment interviews by telephone for logistical reasons (35/701 or 4.4% of follow-up interviews). We paid participants up to a total of $160: $40 for the first interview/training, $25 for each of the two office visits in the first month, and $35 for each of the interviews at 3- and 6-months post-enrollment.

Using a factorial design to test H1a, b, and c is one of the most statistically efficient ways of testing two interventions simultaneously (Piantadosi, 2005). Per Table 1, the main effects of EMI for H1a are evaluated by comparing the rows as (a and b) vs. (c and d); the main effects of EMA for H1b are evaluated by comparing the columns as (a and c) vs. (b and d). The interaction of EMI and EMA is tested by comparing the diagonals as (a and d) vs. (b and c). Each of these comparisons has twice the sample size of a traditional 4-group design. However, if there is a significant interaction, interpretation requires that the data be reanalyzed as a 4-group design (Piantadosi, 2005) by comparing groups a vs. b vs. c. vs. d. This study was designed to have 99% power for a two-tailed test of p<.05 and d=.22 for the proposed factorial design if there was no additional interaction, and 90% or more power for a more conservative analysis of variation for the 4-group test (80%+ with a Bonferroni correction) when there was an interaction.

Table 1.

Factorial design.

		EMA		Main effect of EMI tested with rows as
		No	Yes
EMI	No	a. Control	b. EMA only	a and b
	Yes	c. EMI Only	d. EMA/ EMI	c and d
	Main Effect of EMA tested with columns as	a and c	b and d	Interaction effect tested with diagonals as (a and d) vs. (b and c)

The absence of an interaction in a factorial design does not rule out all possible indirect effects (O’Rourke & MacKinnon, 2018, 2019; Sobel, 1982). Exploratory evaluation of the indirect effect for H2 uses a subset of the data and focuses on whether: H2a) the pattern of EMI utilization will predict more days of abstinence (i.e., establishing a dose response relationship), and H2b) EMI utilization (as defined by H2a) is higher in the group with EMI/EMA than the group with EMI only (groups d vs. c in Table 1). More details on the design are available in the study protocol (Scott, Dennis, & Gustafson, 2017).

2.2. Participants

Eligibility criteria were: (1) met Diagnostic and Statistical Manual Version 5 (DSM-5; American Psychiatric Association, 2013) criteria for one or more SUD in the year prior to treatment intake, (2) engaged in treatment (i.e., received at least 2 sessions of outpatient or 2 nights of residential or 2 doses of medication treatment within 6 weeks of intake; NCQA, 2020), (3) discharged from residential treatment, if applicable, (4) able to communicate in English, and (5) cognitively able to provide informed consent. For logistical reasons, we deemed individuals ineligible if they: (1) currently lived outside Chicago or planned to live outside Chicago during the 6 months of their study participation, (2) expected to be in jail, prison, or another setting that would prevent their use of smartphones, (3) were unable to use a smartphone because of a disability or health condition, (4) were unwilling to learn how to use a smartphone or to complete a survey using a smartphone, (5) were initially admitted to an SUD treatment program that provided intensive recovery services as part of their usual services (e.g., case management, recovery coach), (6) had an existing recovery coach and had been in contact with the recovery coach in the 30 days prior to randomization, (7) failed the Short Blessed cognitive impairment test (Titus et al., 2012), (8) had ever been diagnosed with or told by a physician that they have schizophrenia and/or bipolar disorder, and/or (9) were under the age of 18.

Research staff screened and recruited participants from two treatment agencies in Chicago that provided multiple levels of care. Candidates were screened for eligibility at intake (regarding past-year SUD, residence in Chicago, and ability to participate in study) to these agencies. If they met these criteria, staff told the candidates about the study and asked them to sign a consent for a researcher to contact them upon verification from the treatment program that an eligible candidate had met the additional criteria of having “engaged” in treatment. If applicable, they also had to have been discharged from residential treatment. Researchers then contacted study candidates to verify follow-up contact information and invited them to complete the informed consent and locator documents, followed by randomization to one of the four groups. We present participant characteristics in the Results section.

2.3. Interventions

After randomization, all participants received a 1-hour session on relapse prevention. Depending on assignment, they also received a 1-hour training on how to use the phone and to access the various EMIs and/or a 1-hour training on how to use the phone and to open and respond to the EMAs. Research staff led the trainings, with 1–5 participants approximately weekly at the research office. All participants also came to the research office twice during the first month after enrollment. For participants in the experimental groups the goal was to ensure that they maintained competency to use the apps and to receive refresher training, if needed; individuals in the control group completed a phone survey about their phone use during their visit. At months 3 and 6, all participants completed a comprehensive assessment and urine screen. A different protocol manager managed each of the four conditions, to avoid contamination. Below are details about the non-study related treatment/recovery support services participants received in each of the four conditions during the study period, including data on rates of utilization.

2.3.1. Nonstudy related treatment/recovery support

At any point in the study, participants in all groups could access community-based treatment and/or recovery support services on their own. At the point of random assignment, approximately 45% were receiving outpatient services or methadone treatment and 55% had been discharged from treatment. For the latter, the discharge practice at the participating treatment programs was to provide a recovery plan and referrals to recovery services in the community, such as 12-step groups or other recovery-oriented groups. Six months (~182 days)postrandomization, the 401 participants attended nonstudy self-help meetings on an average of 53 days (29% of 180 days; 81% 1+ days), received an average of 33 days of SUD treatment (18% of 180 days; 49% 1+ days), received an average of 8 days of medication-assisted treatment for alcohol or opioids (4% of 180 days; 13% 1+ days), and stayed an average of 29 days in some kind of controlled environment, such as a hospital or jail (16% of 180 days; 59% 1+ days). There were no statistically significant differences hypothesized or found (at p<.05) in any of these rates of nonstudy services by condition.

2.3.2. Ecological Momentary Interventions (EMIs)

Prior to this study, the A-CHESS (Gustafson et al., 2014) included some education materials about addiction and recovery, reminders about motivators and healthy coping mechanisms, a healthy activities calendar, distractions from craving, and management of negative affect through games and social networking (Heron & Smyth, 2010; McTavish, Chih, Shah, & Gustafson, 2012; Ouimette et al., 2001). Given the current study’s focus on self-managed recovery support (vs. counselor-managed continuing care) and aim to separate out the effects of EMI, EMA, and their combination, we made several adaptations to the existing A-CHESS system. We disabled the A-CHESS applications related to working with a counselor and the daily (1 question) and weekly (10 question) survey. Following a relapse prevention model (Witkiewitz & Marlatt, 2004), we also added several applications to facilitate relapse prevention. To help address risky situations, we added emergency hotlines, treatment locators, and links to free in-person/on-line self-help meetings. To provide support for coping with risky affective states or situations, we added more applications related to relaxation, mindfulness, music, games, and physical and mental exercise. To help maintain healthier lifestyle choices, we added applications to help organize schedules and connect to recovery literature. In this context, we also changed three existing A-CHESS components to focus on variables that supported recovery or triggered relapse, and we set up the “discussion groups” by condition (to avoid contamination) and reframed them as recovery support groups. EMI utilization was high, with an average of 84.43 EMI uses per week (94% using 1 or more EMI per week).

2.3.3. Ecological Momentary Assessments (EMAs)

The phone delivered EMAs randomly 5 times a day within a range of 16 hours selected by the participant. EMAs consisted of a set of 28 questions already described at length elsewhere (Scott, Dennis, & Gustafson, 2018). Participants recorded their recent substance use and exposure within the prior 30 minutes to internal and external protective and risk factors (i.e., people, places, activities, and feelings); participants then rated the extent to which these factors supported their recovery, made them want to use drugs or alcohol, or had no impact. Although the questions and answers remained the same, the order of the sections, questions within sections, and categorical responses within questions varied in random order. Each EMA took 2 to 3 minutes to complete and once prompted, participants had up to 30 minutes to respond. After completing an EMA, participants received a “thank you” response.

Participants were excused from completing EMAs for one of two reasons: 1) when they were unable to use a phone because they were participating in a treatment session, working, in class for school, incarcerated, ill, attending a self-help meeting, and so on; or 2) their phone was unavailable or not operating (including because it was lost or broken, or the system was temporarily down). When staff determined scheduling conflicts a priori, the EMAs would be set up to be recorded as excused automatically. Other EMAs were classified as excused after they occurred. Protocol monitors contacted participants when 2 consecutive EMAs had been missed to learn about the circumstances leading to the missed EMAs. In these situations when staff updated records for any missed EMA after the fact (e.g., found out that someone had been incarcerated or hospitalized), the second author also reviewed the changes before it was finalized. Of the 167,542 randomly scheduled times participants were prompted to complete an EMA, 41,718 (25%) were excused because the phone was lost/stolen, broken or the system was down (23%); the person was at treatment (22%), work (19%), temporarily incarcerated (14%), sick or ill (5%), at a self-help meeting (1%), or in school (1%) at the time of the prompt; or for other less common reasons (14% combined). Of the remaining 125,824 random times EMAs had been scheduled, EMAs were completed 99,185 (79%) times and were not completed 26,639 (21%) times.

2.3.4. EMIs + EMAs combined

Participants in the EMI + EMA condition received both of the interventions we have described. In addition to the “thank you” after each completed EMA, they received a message related to their relapse risk (high, moderate, or low) and encouragement to use an EMI. Based on a previously reported analysis (Scott et al., 2018), we assessed relapse risk using a participant’s EMA responses from the last 7 days and classified as: (1) high if there was substance use in the past 30 minutes, (2) moderate if there was substance use in the past week, or (3) low if it was more than a week since last substance use was reported. Each level of risk has a corresponding set of messages; one message from the appropriate set was randomly selected and sent to the participant after the EMA was completed. Below this risk-adjusted message, participants always saw the menu of EMIs. This combined condition had more EMI utilization than the EMI only group (96% vs. 92% of weeks with 1+ EMI, OR=1.92, p<.05) and more EMA completions than the EMA only group (80% vs. 77%, OR=1.20, p<.001).

2.3.5. Control condition

The fourth group did not receive a phone, data plan, or access to the A-CHESS suite of EMI applications or EMA.

2.4. Measures and data sources

2.4.1. Primary outcome

We based the primary outcome measure for H1 and H2 on the question, “During the past 90 days, on how many days did you go without using any alcohol or other drugs?” We asked this at baseline just prior to randomization, as well as at 3 and 6 months later, with an interclass correlation coefficient (ICC) of .42 for observations within participants in this study.

2.4.2. Secondary outcome

For the indirect effect of EMA via EMI utilization for H2, we looked at the weeks with 1+ EMI (ICC=.42), as well as the number of EMIs used per week overall (ICC=.43), by 5 EMI types in the study-specific version of the A-CHESS app (e.g., reach out to others, relax, learn, recovery motivation, and exercise) and by 4 to 8 specific EMIs subtypes within each type (broken out further in Results). In each case, we based “use” on clicking on a link to start an EMI subtype, then EMI subtypes summed to the respective EMI types, and then summed across EMI types to the overall total. Note that this count did not include looking at the home screen, submenus, settings, EMA notices, or thank you messages—it includes only the initiation of a specific EMI.

2.4.3. Interview data

We collected self-reported data on participant characteristics, services received, and outcomes with the Global Appraisal of Individual Needs Quick Version 3 (GAIN-Q3) (Titus et al., 2012). The 5- to 6-item short screeners in this instrument have been shown to be correlated .9 or more with the 16- to 41-item longer version of the full GAIN. Reporting 2 or more SUD symptoms and 1–3 days of use had high rates of sensitivity (82–96%), specificity (83–96%), and kappa (.60 to .89+) as measures of SUDs for alcohol, cannabis, stimulants, opioids, and/or across any substance (Dennis, Modisette, & Estrada, 2020).

The Cronbach’s alphas in this sample were similar to the published adult norms (GAIN Coordinating Center, 2013) for the 5- to 6-item screeners related to: 1) SUD (alpha=.9 vs. .9); 2) internalizing mental health disorders (alpha=.8 vs. .8); 3) externalizing mental health disorders (alpha=.7 vs. .8); 4) HIV risk behaviors and victimization (.7 to .7); 5) health problems (.6 to .7); 6) crime/violence toward others (.5 to .7); and 7) other sources of stress screener (.5 to .6). Copies of the instrument, reports, manual, scoring syntax, norms/psychometrics, and publications are publicly available on-line at www.gaincc.org. Interviews were conducted at enrollment, and 3 and 6 months postenrollment. The GAIN Coordinating Center trained and certified all staff on the use of the GAIN to maximize reliability and validity. Interviewers were blind to assignment.

2.4.4. Urine drug screen and validation

We supplemented interviews with onsite urine screens. We tested urine onsite with CLIAA-waived QuikScreen cups using an immunochromatographic assay for rapid (2 to 5 minutes) qualitative results based on the Substance Abuse and Mental Health Services Administration (SAMHSA)-standard cutoffs for alcohol (20 mg/dl or 0.02% BAC), amphetamine/methamphetamine (1000 ng/ml), cannabis (50 ng/ml), cocaine/benzoylecgonine (300 ng/ml), and opiates/morphine (2000 ng/ml). Following the NIH’s Phenx Toolkit recommended protocol (Scott & Dennis, 2009), we conducted urine tests onsite and prior to administering the drug use questions. The false negative rate for self-report compared to all available sources of data was 8% (kappa=.84), with no significant difference by condition.

2.4.5. EMI and EMA data

The EMI records data include a URL for each menu and EMI application link within A-CHESS, whether on the phone or on a website, as well as a date/time stamp. Participants had access to A-CHESS and other EMIs 24 hours a day, 7 days a week, with participants averaging 2,195 EMI uses over 6 months. The EMA data include the date/time it was due and completed (or timed out), the completion status, the survey responses, and a geocode for where the person was located when they initiated the app. The EMA has been described at length elsewhere (Scott et al., 2018).

2.5. Randomization

We used urn randomization (Charpentier, 2003) with a base rate of 25% per condition stratified by participant characteristics (gender, race, age, recruitment cohort, level of comfort using a smartphone), pretreatment measures of the dependent variables (days of abstinence), and prerandomization treatment (level of care, length of stay, type of discharge/treatment status) to increase the likelihood that participants are distributed similarly across conditions. Urn randomization adjusts the probability of assignment to a condition in ways that simultaneously minimizes differences in multiple stratification variables (Stout, Wirtz, Carbonari, & Del Boca, 1994; Wei & Lachin, 1988). Only the research coordinator and protocol monitor responsible for scheduling, proficiency testing, and protocol supervision had access to information about assignment. The A-CHESS server controlled delivery of EMI, EMA, or both.

2.6. Data collection procedures

Chestnut’s follow up procedures used in this study have previously produced more than 90% follow-up rates across studies involving more than 40,000 patients of varying diagnoses and sociodemographic characteristics for follow-up periods ranging from 3 months to 15 years (Scott, 2004). These procedures ensured contact for participants in all study arms including controls. Participants in all four groups also attended two follow-up meetings during the first month. For participants in the experimental groups, the meetings were to ensure proficiency of use of the phone and applications. Individuals in the control group also came into the office and completed a phone survey about their phone and text use. At 3 and 6 months after randomization, all participants completed a 1-hour interview to measure outcomes. Participants completed most of the interviews in person (95%) and used cloud-based GAIN ABS software (91%) with built in automated skip outs, range checks, and validity checks. In the remaining 4.4% of cases in which face-to-face interviews were infeasible, we conducted interviews by phone or on a hard copy instrument and then they were keyed. Participants completed urine tests at all in-person interviews and we documented their results in a local Excel file. We tracked EMI completion and EMI utilization online.

2.7. Analysis

We conducted all analyses using IBM SPSS version 25.0. For hypotheses 1, we used a 2 X 2 factor repeated measures analysis of variance (or repeated measures general linear model analysis) with the days of abstinence at each time point as the dependent variable. The 7 fixed predictors in the model included time (0, 3, and 6 months postenrollment); the main effect of EMIs (rows from Table 1); the main effect of EMAs (columns from Table 1); the interaction of EMI x EMA (diagonals from Table 1); and the interaction of Time x EMA, Time x EMI, and Time x EMA x EMI. The within subjects’ factor was the time effect, looking at the 3 waves within person. The hypothesized outcome for H1a is tested with Time x EMI; the hypothesized outcome for H1b is tested with Time x EMA; and the hypothesized outcome for H1c is tested with Time x EMI x EMA. The results (i.e., values, significance levels, effect sizes) did not change whether the small amounts of missing data were replaced with the average of the surrounding observations within person or not.

To evaluate the indirect effect of EMA (via EMI utilization) on days abstinent (Hypothesis 2a & 2b), we subset to the two EMI groups. Because we had multiple types of EMI and summary levels that were by definition correlated, H2a focused on identifying a simplified composite measure with stepwise regression to predict the days of abstinence outcome. We entered each EMI predictor variable sequentially based on its probability of a type 1 error (alpha), starting with the lowest value and stopping when no others were less than .05. One at a time, variables were then removed based on their probability of a type 2 error starting with the highest value and stopping when no other were equal to or greater than .06. This procedure was repeated until no other variables were available to enter or remove based on these cut points.

To test H2b, we constructed a path model where the randomly assigned condition (EMI only or EMI+EMA) was the main exogenous variable, the composite measure of EMI utilization (from H2a) was the proximal outcome/mediator, and the average days of abstinence across 3 and 6 months was the distal outcome. If there was only one observation, it was used in order to use all available data (vs. list wise deletion). We then utilized the Sobel test with bootstrapping (Preacher & Hayes, 2008) to test for significant indirect effects. We used the conventional p < .05 to define statistically significant relationships. We also reported Cohen’s f-index effect size based on the eta-squared (Ferguson, 2009). We interpreted an effect size of 0.10 as small, 0.25 as moderate, and 0.40 or more as large.

3. Results

3.1. Pre- and postinclusion case flow

Figure 1 shows the study CONSORT chart. Of the 4,809 people we screened at treatment admission, 80% (3,866/4,809) were not in the target population prior to randomization. The (overlapping) reasons for exclusion included: 1,213 lived outside of the city of Chicago, 819 were admitted to programs that already provided an alternative intensive recovery support, 737 were planning on moving outside of Chicago in the next 6 months, 447 could not be contacted about participation with the information provided, 266 were not comfortable answering short surveys on a cell phone (required for 2 of the 4 conditions), 219 were already receiving recovery coaching services, 199 received fewer than 2 treatment sessions within 6 weeks (i.e., did not engage in treatment), and 20 did not initiate treatment after the intake appointment/screening. Of the remainder, 41% (394/943) refused to participate at some point and an additional 16% (148/943) agreed and were invited to training but did not show.

Figure 1.

Smartphone Recovery Support Services (SRSS) Experiment CONSORT Chart

* Denominator is the number of follow-ups due minus the number of participants that died.

** The Ns reported here are the sample sizes used in the analyses: participants with Intake, 3 and 6-month follow-ups.

*** The denominator is the total observations of the participants in the research office during week 1, week 2 visits, plus 3 and 6-month follow-ups.

Of the remaining eligible individuals, 401 agreed to participate and we randomized them: we assigned 100 to the EMI-only condition, 98 to the EMA-only condition, 105 to the combined EMA and EMI condition, and 98 to the control condition. The next set of rows shows the data collection at enrollment (all 100%), 3 months (96–100%), and 6 months (94–100%) postenrollment, as well as the rate of completing all 3 interviews (93–99%), which we required for inclusion in the analyses presented here. Of the multiple in-person meetings where we planned urine analysis, participants completed urine tests on 90–94% per condition.

Of the 100 in the EMI-only condition, 54 averaged 70 or more EMIs per week (full), 46 averaged 1 to 69 EMIs per week (partial), and no one averaged 0 EMIs per week (none). Of the 97 people in the EMA-only condition, 66 completed 80% of their EMAs of up to 910 planned EMAs (full), 30 completed 1 to 79% of their EMAs (partial), and 1 completed no EMAs (none). Of the 105 in the combined EMA/EMI condition, 47 met the criteria for full use of both EMAs (80%+ completed) and EMIs (70+ weeks), 54 met criteria for partial use of at least one of them, and 4 had no EMA or EMI use.

3.2. Participant characteristics

Participants were predominately male (61%), African American (70%), had a mean age of 44, had a high school degree or equivalent (67%), and had used alcohol and/or other drugs on 13 or more of the 90 days prior to randomization (60%; which may include pre-, during, and post-treatment). Although study enrollment was an average of 64 days after the initial treatment intake, 56% still self-reported substance use and met 2 or more criteria for SUDs in the prior 90 days; this included self-reported use and past 90-day criteria for any kind of alcohol (34%); stimulants, including cocaine (32%); opioids, including heroin (31%); cannabis, including hashish (18%); and other drugs (6%). Moreover, 38% self-reported diagnostic criteria for more than one SUD and substance use in the prior 90 days. Many also self-reported problems (1 or more days and 2 or more symptoms) related to internalizing or externalizing mental health (57%), stress (52%), physical health (44%), high-risk behaviors/victimization (39%), and/or crime/violence (9%). Furthermore, multiple problems were the norm with a mean of 3.2 of these SUDs/other problems and 59% with 3 or more SUDs/other problems.

Participants received their most recent service from a mix of residential (36%), intensive outpatient (32%), recovery homes (11%), outpatient (8%), and detoxification (1%); their mean length of stay from intake to treatment engagement was 46 days. At the point of randomization, 45% were still receiving some type of service (e.g., methadone maintenance, intensive outpatient, outpatient, recovery home) and 55% had already been discharged. The mean time from the date of treatment engagement to the date of enrollment/random assignment was 18 days. Table 2 shows each of these clinical and prerandomization treatment characteristics overall and by condition. None of these rates varied significantly by condition at p<.05. Taken together, these characteristics demonstrate that high rates of co-occurring SUDs and multi-morbidity were the norm in this sample, and that participants had a continuing high need for further post-treatment intervention.

Table 2.

Participant characteristics and services prior to randomization.

			Randomly Assigned Group
Characteristics at or during the 90 days before enrollment and randomization		Total (n=401)	Control (n=98)	EMI (n=100)	EMA (n=98)	EMA (n=105)	Chi-sq.	p
Female		39%	34%	42%	39%	41%	1.72	0.631
Race	African
	American	70%	68%	74%	69%	68%	7.88	0.552
	Caucasian	20%	18%	16%	21%	25%
	Hispanic	6%	6%	8%	4%	5%
	Other/Mixed	4%	7%	2%	5%	3%
Age	18 to 25	6%	2%	8%	7%	5%	15.79	0.069
	26-39	26%	30%	21%	29%	26%
	40-49	33%	31%	31%	25%	44%
	50+	36%	38%	40%	40%	26%
Mean Age (standard deviation)		44.2 (11.0)	44.5	45.1	43.6	43.6	1.64	0.650
High School Graduate or GED		67%	74%	62%	65%	69%	3.23	0.353
13 or more Days of Substance Use \a		60%	59%	61%	53%	65%	2.99	0.390
Any Past 90 Day Substance Use
Disorder (SUD) \b		56%	55%	60%	53%	54%	1.12	0.777
Alcohol Use Disorder \c		34%	35%	33%	34%	34%	0.07	0.996
Stimulant Use Disorder\d		32%	32%	32%	34%	31%	0.14	0.990
Opioid Use Disorder \d		31%	26%	37%	31%	32%	3.11	0.380
Cannabis Use Disorder \e		18%	21%	18%	14%	17%	1.75	0.641
Other Drug Disorder \d		6%	6%	8%	7%	4%	1.73	0.644
Multiple Substance Use Disorders		38%	39%	39%	39%	37%	0.10	0.994
Mean Count of SUD (Stand. Dev.)		1.2 (1.3)	1.2	1.3	1.2	1.2	0.51	0.918
Mental Health Problems\f		57%	64%	51%	63%	51%	6.48	0.092
Stress Problems\f		52%	53%	43%	55%	55%	4.08	0.254
Physical Health Problems\f		44%	43%	46%	51%	38%	3.63	0.306
Risk/Victimization Problems\f		39%	41%	37%	40%	40%	0.35	0.954
Crime/Violence Problems\f		9%	11%	10%	8%	8%	1.00	0.802
Count of Substance Use and Other	0	11%	12%	12%	7%	13%	3.67	0.935
Problem Areas during the 90 days	1	15%	13%	16%	16%	13%
before enrollment and	2	15%	13%	14%	17%	17%
randomization\g	3-10	59%	61%	58%	59%	56%
Mean Count of Problems (SD)		3.2 (2.3)	3.3	3.2	3.4	3.1	0.80	0.849
Discharged or	Residential	36%	35%	37%	34%	40%	13.53	0.567
Last service from:	Intensive OP	32%	39%	28%	35%	29%
	Recovery Home	11%	14%	12%	9%	10%
	Methadone	10%	6%	13%	12%	10%
	Outpatient	8%	4%	9%	7%	10%
	Detox	1%	2%	1%	3%	0%
Still In Treatment At Randomization \h		45%	49%	41%	46%	45%	1.31	.729
Mean Days / Length of Stay of Substance
Use Treatment (Standard Deviation)		46 (37)	47	45	48	44	0.71	0.871
Mean Days from Substance Use Treatment
Discharge/Last Service to Random
Assignment (Standard Deviation)		18 (23)	15	20	18	17	2.20	0.532

^\aDuring the past 90 days (weekly approximated as 13 or more of 90 days).

^\bAny SUD on Q3 based on self-report of any days of alcohol or drug use and 2 or more substance use disorder symptoms on the substance disorder screener in the past 90 days.

^\cAlcohol use disorder required 3+ days of use or any intoxication in past 90 days and 2 or more symptoms on the substance disorder screener in the past 90 days.

^\dSpecific DSM 5 SUD required any use of substance in past 90 days and 2 or more symptoms on the substance disorder screener in the past 90 days.

^\eSpecific DSM 5 cannabis use disorder required 3+ days of cannabis use in the past 90 days and 2 or more symptoms on the substance disorder screener in the past 90 days.

^\fSpecific problems required any days of problems and 2+ symptoms on the corresponding screeners.

^\gCount of estimated individual substance use disorders and the other problem areas: mental health, physical health, stress, risk behaviors and crime/violence.

^\hIn treatment based on the last service within 7 days of random assignment.

3.3. Are there direct effects of time, EMI, EMA, and their interactions?

Across groups there was a small but statistically significant direct effect of Time across conditions (F_(2,734)= 4.33, p=0.014, Cohen’s effect size f= 0.11, 95% C. I. [0.03, 0.18]). Across groups, days of abstinence increased from 60 days at baseline to 65 days at 3 months and 63 days at 6 months (all with regard to past 90 days).

For H1a, there was a direct effect of “time-by-EMI” utilization (F_(2,734= 4.85, p=.009, effect size f=0.11, 95% C.I. [0.02, 0.18]), with those assigned to the EMI row of the study design as shown in Table 1 increasing and maintaining their days of abstinence over the 3 time periods (means=58 to 66 to 66 days). In contrast, for those assigned to the No EMI row of Table 1, their days of abstinence at 6 months was fewer than any other time point (means=62 to 63 to 59 days). Figure 2 shows this difference standardized for both groups and all 3 times as z-scores based on the grand total mean and standard deviation at baseline.

Figure 2.

Days Abstinent from Alcohol or Other Drug Use z-scores by EMI

*z-score based on the mean and standard deviation of the total sample at the point of randomization.

For H1b, there was not a significant “time-by-EMA” direct effect (F_(2,734)=1.54, p = .216, f=0.06; columns in Table 1). For H1c, there was not a significant “time-by EMI-by-EMA” interaction effect (F_(2,734)=0.80, p=0.447, f=0.04; diagonals of Table 1). None of these findings changed with replacement of the small amount of missing data by the average of adjacent observations within subjects.

3.4. What is the relationship between EMI utilization and days of abstinence?

To address exploratory H2a, we subset to the two groups in the Table 1 row with access to EMI (i.e., EMI only and EMI+EMA). To increase power, we looked at the relationship between EMI utilization and abstinence using two different observation periods: 1) between baseline and 3 months, and 2) between 3- and 6-months postbaseline. We then took the average of these two observations to reduce missing data and increase the stability of the estimates. Table 3 provides the means and standard deviations of the past 90 days of abstinence averaged across the 3- and 6-month postbaseline periods (first row), two summary measures of any EMI use (proportion of weeks with any EMI use and number of EMIs used), 5 types of EMIs (reach out to others, relax, learn, recovery motivation, and exercise), and 4 to 8 subtypes of EMIs within each. Within a few of the EMI subtypes, more than one EMI provided similar types of intervention activities (e.g., listen to music comprised different EMIs for 8-track, Pandora, Sound track, Spotify, Tune In Radio) that are not broken out here. The middle of the table shows univariate relationships between each of these measures of EMI utilization and days of abstinence after controlling for repeated observations by person.

Table 3.

Relationship with days of abstinence and various measures of EMI use at 3 and 6 months.

Overall/EMI Topic/EMI type*	Average across the 3- and 6-month observations N = 200		Univariate Results		Multivariate Stepwise Results
	Mean	Std. Dev.	Beta	p-value	Beta	p-value
Days abstinent from alcohol or other drug use in the past 90 days	65.74	28.30
Proportion of weeks 1+ EMI accessed	0.94	0.14	0.22	0.002	0.19	0.007
Total EMI accessed	872.97	657.99	0.18	0.013
Reach Out to Others	614.29	502.12	0.13	0.060
Contact your sponsor	4.78	9.45	0.15	0.039
Text your friends	1.76	2.38	0.05	0.481
Discussion groups	169.59	201.75	0.10	0.165
My messages	224.69	159.56	0.11	0.119
Support team	61.00	58.22	0.04	0.603
Facebook	138.51	297.16	0.08	0.239
My Friends	4.26	5.59	0.01	0.925
Find meetings	9.70	9.60	0.09	0.195
Relaxation	206.50	277.65	0.16	0.026
Listen to music	150.77	233.01	0.19	0.007	0.16	0.027
Listen to guided relaxation	10.57	15.82	0.03	0.704
Listen to recovery stories	9.08	11.16	0.01	0.874
Play games	36.07	89.71	−0.01	0.851
Learn	12.19	14.62	0.05	0.474
Recovery Q and A	2.83	5.52	0.04	0.613
Recovery articles	3.14	5.03	0.05	0.493
Recovery website	1.41	2.19	0.04	0.605
Read Big Book of AA	2.22	3.07	0.12	0.083
Daily Recovery app	2.48	4.00	−0.05	0.509
Recovery information	0.11	0.42	0.07	0.341
Recovery Motivation	24.24	26.33	0.01	0.893
Recovery motivation page	15.38	16.02	0.03	0.715
Words	2.83	5.79	−0.04	0.565
Pictures	1.49	2.41	−0.02	0.747
Daily Prayer	2.96	5.53	0.04	0.578
Sobriety Counter	1.57	3.16	−0.03	0.692
Exercise	15.75	28.25	0.11	0.120
Physical Exercise	7.05	12.44	0.13	0.067
Daily Workout	1.04	1.14	0.03	0.644
Google Fit	0.93	1.33	0.15	0.042
S Health	5.81	14.80	0.08	0.240
Stretching routines	0.91	1.02	0.038	0.600
					R-sq.	0.072

Note: Bold p<.05

^*includes more than one app in a few cases

Of the 35 variables in Table 3, 6 (17%) were significantly related to days of abstinence, including the proportion of weeks with 1+ EMI, the total number of EMIs, one of the reach out to others EMIs (contact your sponsor), the relaxation intervention and 1 of its EMIs (listen to music), and the google fit EMI. Because these measures are highly correlated, we also conducted a regression with stepwise variable selection. The last two columns show the result of the simplified model using just two variables. These two variables (proportion of weeks using EMIs and the frequency of listening to music via EMI) predicted 7.2% of the variance in days of abstinence (F_(2,195,)=7.56, p<.001, 95% C.I. [0.01, 0.14]).

3.5. Are there indirect effects of EMA, via EMI, on days of abstinence?

While interactions were not found in the 2 × 2 factorial design, we based our exploratory hypothesis (2b) on the assumption that we could reasonably expect the 5-times-a-day EMAs with reminders to use EMIs to increase EMI utilization and thereby have an “indirect” effect on days of abstinence. Since assignment to EMI only or EMI+EMA was random, this question can be analyzed as an experiment within an experiment. Relative to the EMI only group, the combined EMI+EMA group had significantly more weeks with at least 1 EMI accessed (92% vs. 96% of weeks; F₍₁,₁₉₉₎ = 4.11, p = .044, Cohen’s f = 0.14, 95% C.I. [0.00, 0.28]). In an average week, participants in the combined group were significantly (p<.05) more likely to access 5 different types of EMIs, including: my friends (Cohen’s effect size f=0.23), stretching routines (f=.18), recovery motivation (f=.17), reading big book of AA (f=.16), and text your friends (f=0.15).

Figure 3 is a path model showing the indirect relationship of being assigned to the combined EMI+EMA group on EMI utilization and EMI utilization on days of abstinence during the past 90 days. The first box on the left is 1 for those assigned to the combined EMI+EMA condition and 0 for those assigned to EMI only. The middle box in Figure 3 is based on the stepwise analysis in Table 2; we computed a composite measure of EMI utilization using the following equation: EMI Utilization = 26.21 + 39.20*(weeks 1+ access) + 0.02*(listen to music). The right box in Figure 3 is days of abstinence, which is the average of the past 90-day outcome observations at 3 and 6 months. We averaged predictor variables (i.e., EMI utilization) over the two time periods and we averaged the outcome variables (i.e., days of abstinence) to use all available data (vs. list wise deletion if we used the total). The numbers above the arrows are the significant direct path effects (indicator of effect size). The significant percentage of variance is explained above the boxes. Using the Preacher and Hayes method (2008), this path model produces a statistically significant (albeit small) indirect effect of .15 × .27=0.041.

Figure 3.

Indirect effect of EMA (via EMI) on Days of Abstinence over Past 90 Days

3.6. Serious and other adverse events

Four participants died during the 6-month study period and several were admitted to an emergency department or hospitalized for health reasons (30%) or mental health reasons (8%). Fourteen percent reported one or more days in jail or prison. In addition, there were a total of 423 person-days with self-reported adverse events (AE) (e.g., behavioral problems, health problems, mental health problems, substance use-related problems, trauma-related problems, illegal activity, and victimization), for an average of 4.3 serious adverse events (SAE)/AEs per person. None of the SAE/AEs appeared to be related to the research procedures, there were no significant differences in number or type of SAE/AEs by condition, and these SAE/AE rates are similar to or lower than the rates prior to randomization, as reported in the baseline assessment. A detailed table of the results is available from the first author.

4. Discussion

We found that self-initiated EMIs delivered by smartphone had a small (Cohen effect size f=.11) but statistically significant effect on increasing the number of days abstinent for people following engagement in SUD treatment (per Figure 2). It is particularly interesting that the consistency with which individuals used EMIs (proportion of weeks using EMI) was a better predictor of abstinence than the simple number of EMIs used (quantity).

The effect of the type of EMI utilized was mixed. Only 1 in 5 had a significant univariate relationship with days of abstinence. Moreover, in the stepwise regression, after we entered the proportion of weeks used (i.e., consistency of use), only 1 EMI—playing music—had additional predictive value. These findings are consistent with a previous study on EMI (Gustafson et al., 2014) in terms of a) the small positive effect of providing recovery support via EMIs post-SUD treatment, and b) the stronger effect related to the proportion of weeks EMIs are used rather than to the number or specific EMIs used.

This study advances findings from the prior study (Gustafson et al., 2014) in terms of: a) impacting abstinence from alcohol and other drugs (vs. just alcohol use), b) being self-managed vs. counselor supported or managed, c) having higher rates of EMI utilization, and d) demonstrating univariate and multivariate effects of specific EMIs. It is also notable that the additional effect of EMI vs. no EMI reported here was close to the median effect of SUD treatment vs. no/minimal treatment in prior meta-analyses (Cohen’s effect size f = 0.11 vs. 0.13) (Prendergast, Podus, Chang, & Urada, 2002). Moreover, this effect was demonstrated above and beyond the effects of treatment engagement.

Although the utilization of EMAs did not have a direct effect on days of abstinence, it was associated with increased and more consistent EMI use and thus had an indirect effect on days of abstinence. Additionally, our study was one of the first to investigate the combined effects of EMI and EMA. In particular, the finding that EMA assignment increased the utilization of a variety of EMIs suggests that looking at the EMA response patterns at the individual level may provide better predictions of abstinence and the specific EMI that increases abstinence outcomes.

4.1. Strengths and limitations

The study has several strengths including that it is theory-based; has strong measurement; uses randomization; builds upon an existing EMI suite and infrastructure; simultaneously measures multiple EMIs and risk factors at each EMA, which yielded a large dataset of temporal observations across study participants; and has high-fidelity implementation (as seen in the measures of EMA/EMI utilization). Limitations include a small sample size, limited number of recruitment sites and communities, as well as a limited follow-up duration (6 months). The latter is important because EMI utilization decreased over time and the slope of the decline varied by EMI type and subtype. In addition, the study did not track participants’ healthcare utilization, including their utilization of SUD treatment services and medications for SUD postrandomization. Study findings are limited in generalizability to the unique sample of participants and programs from which they were sampled, i.e., programs that did not already provide intensive post-treatment continuing care. While the current focus of this study is on reporting the main findings from a clinical trial, additional within subject and/or over time analyses could be done with the EMI and EMA data, as well as moderator (e.g., type of SUD, prior levels of care) and mediator (e.g., other treatment or support Services received) analyses that are worthy of further study.

5. Conclusion

This study found that smartphone-delivered self-initiated EMIs following community-based SUD treatment engagement have a statistically significant additional effect on reducing alcohol and drug use in the 6 months following treatment engagement relative to receiving no EMIs. While EMAs did not directly impact abstinence, they did increase utilization of EMIs and thus have an indirect effect. Study findings have implications within the context of capacity limitations to provide SUD treatment to those with indicated need, as well as continuing care interventions following primary treatment. National survey data consistently show that only about 10% of the approximately 20 million individuals who meet clinical criteria for SUD actually receive any treatment for these disorders in a given year (Park-Lee, Lipari, Hedden, Kroutil, & Porter, 2017). Rates are even worse for adolescents and young adults. Our understanding of the utility of smartphone-based EMIs may be improved by further research that evaluates the efficacy of EMIs for individuals seeking to change their substance use outside of SUD treatment or through other service sectors (e.g., primary care), thereby expanding access to effective interventions to a broader population. In addition, research that analyzes the effects of the individual elements of EMI or that uses EMA to predict intervention need and to push specific mobile app-based interventions may improve the utility of EMI to address SUD.

Highlights.

• Smartphone interventions can aid in recovery following treatment for substance use

• Ecological momentary interventions (EMI) and assessments (EMA) were reliably delivered

• EMI (with or without EMA) significantly increased days of abstinence over 6 months

• Path model showed a small significant indirect effect of EMAs on abstinence via EMI

• Self-managed utilization of EMI and EMA has potential for increasing recovery support