Abstract
The Therapeutic Education System (TES), an Internet-version of the Community Reinforcement Approach plus prized-based motivational incentives, is one of few empirically-supported technology-based interventions for substance use disorders. To date, however, there has not been a study exploring differences in substance use outcomes or acceptability of TES among racial/ethnic subgroups. This study uses data from a multi-site (N=10) effectiveness study of TES to explore if race/ethnicity subgroups (White [n=267], Black/African American [n=112], and Hispanic/Latino [n=55]) moderate the effect of TES. Generalized linear mixed models were used to test if abstinence, retention, social functioning, coping, craving or acceptability differed by racial/ethnic subgroup. Findings demonstrated that race/ethnicity did not moderate the effect of TES vs treatment-as-usual (TAU) on abstinence, retention, social functioning or craving. A three-way interaction (treatment, race/ethnicity, and abstinence status at study entry) showed that among White participants not abstinent at study entry, TES was associated with greater coping scores compared to TAU participants (p=.008); among Black participants abstinent at study entry, TES was associated with greater coping scores compared to TAU participants (p<.001). Acceptability of the TES intervention, although high overall, was significantly different by race/ethnicity subgroup with White participants reporting lower acceptability of TES compared to Black (p=.006) and Hispanic/Latino (p=.008) participants. TES appears to be a good candidate treatment among a diverse population of treatment seeking individuals with substance use disorders.
Keywords: Internet-delivered treatment, technology, substance use disorders, race/ethnicity, acceptability
INTRODUCTION
Technology-assisted interventions for addictions
Technology-assisted interventions for substance use disorders show great promise for providing access to high quality, evidence based interventions while reducing barriers such as cost, geographic distance from specialty care, and stigma (Carroll & Rounsaville, 2010). Systematic reviews of this increasingly emerging modality of treatment indicate inconsistent attention to quality and effectiveness (Wood et al., 2014; Moore, Fazzino, Garnet, Cutter, & Barry, 2011; Bewick et al., 2008), but limited evidence suggests internet-delivered treatment for substance use results in greater knowledge, better retention and higher rates of abstinence (Moore, Fazzino, Garnet, Cutter, & Barry, 2011; Bewick et al., 2008; Campbell et al., 2014). Such interventions have the added advantage of ensuring high fidelity to empirically supported treatments, with lower demands on staff in terms of necessary training and supervision (Campbell et al., 2012).
Racial and ethnic minorities comprise 40 percent of admissions to publicly funded substance abuse treatment programs (National Institute on Drug Abuse [NIDA], 2011), however recent research suggests these populations may be at particular risk for poor treatment outcomes, largely due to socioeconomic factors (Saloner & Le Cook, 2013). Despite socioeconomic challenges, however, the digital gap among racial and ethnic minorities and Whites has narrowed over the past 15 years (Perrin & Duggan, 2015). Further, racial and ethnic minority groups are more likely to access the Internet on smartphones (compared to computers or other devices) and to take advantage of a wider array of their phones’ data functions (e.g., looking up information about a health condition) compared to White mobile phone users (Smith, 2010; 2015). Given that integration of the Internet into the lives of racial and ethnic minorities, Internet-based interventions have the potential to reduce racial disparities in substance use disorder outcomes by increasing access to high-quality treatment and removing barriers to traditional substance abuse treatment, such as stigma and time conflicts (Gibbons, 2011; Rapp, Xu, Carr et al., 2006). To date there has not been a study exploring differences in outcomes or acceptability of Internet-delivered substance abuse interventions for racial and ethnic subgroups.
Effectiveness of CM and CRA in treating drug addiction
Contingency Management (CM) and Community Reinforcement Approach (CRA) are two of the most empirically supported addiction treatments (Budney & Higgins, 1998; Smith et al., 2001; Stitzer, Petry, & Pierce, 2010). Moreover, research suggests that combining these two interventions has been shown to improve outcomes above and beyond each intervention on its own (Garcia-Fernandez et al., 2011; Higgins et al., 2003). However, Internet-delivered versions of CM and CRA are relatively new. A recent randomized control trial compared treatment-as-usual (TAU) to a computer-assisted Therapeutic Education system (TES), a combination of Internet-delivered CRA plus CM, and found that participants in TES were retained in treatment significantly longer and had a greater abstinence rate than TAU participants (Campbell et al., 2014). Other studies have also shown the effectiveness of automated CM for the treatment of cocaine and opioids (Vahabzadeh, Lin, Mezghanni, Epstein, & Preston, 2009), and the promise of Internet-delivered CM programs in promoting smoking cessation among adolescents and adults (Dallery, Gleen, & Raiff, 2007; Reynolds, Dallery, Shroff, Patak, & Leraras, 2008). Although Internet-delivered approaches have demonstrated effectiveness in treating addictions in general, it is unclear if findings are effective across racial and ethnic minority subgroups.
Racial/ethnic differences in CM and CRA delivered in traditional treatment settings
Several randomized trials conducted in traditional outpatient settings have found racial and ethnic differences in CM and CRA outcomes. For example, in a randomized trial comparing CM to standard care (Montgomery, Petry, & Carroll, 2015), CM was more effective than standard care in improving retention and cocaine use outcomes among White adults who initiated treatment with a cocaine-positive urine sample. However, among African American participants who began treatment with a positive sample, participants in CM did not remain in treatment longer than those in standard care, and cocaine use outcomes were muted relative to White participants. Another study examining the effectiveness of an adolescent version of CRA found that African American adolescents had significantly higher rates of treatment satisfaction than their White counterparts (Godley, Hedges, & Hunter, 2011). The racial/ethnic differences found in traditional CM and CRA suggest that Internet-delivered outcomes might also vary by race/ethnicity.
Purpose
The purpose of the current study was to examine treatment outcomes among racial and ethnic minority subgroups enrolled in a clinical effectiveness trial of computer-assisted TES in outpatient substance abuse treatment. As new technologies are developed to overcome the difficulties in access to evidence-based treatment for substance use disorders, it is important to understand whether such innovative interventions are acceptable and effective across diverse patient populations. In the current study, we sought to describe substance abuse treatment outcomes for racial/ethnic subgroups of patients enrolled in the trial, and to understand whether abstinence from substance use and retention in treatment outcomes varied by race and ethnicity. We further explored a number of additional outcomes associated with theoretical mechanisms of the CRA: social functioning, coping skills, and drug and alcohol craving. Also, we sought to compare the acceptability of TES versus TAU among diverse subgroups of the treatment population. Findings from this study will provide information about the promise of technology-based interventions among diverse outpatient treatment populations.
METHODS
Participants
Participants (N=507) were recruited from 10 outpatient substance abuse treatment programs affiliated with the National Drug Abuse Treatment Clinical Trials Network and located in diverse geographic locations across the United States. Each program enrolled approximately 50 participants (range=38–60) between June 2010 and August 2011. Eligible participants were: (1) 18 or older, (2) using illicit substances in the 30 days prior to study entry (or 60 days if the patient was exiting a controlled environment), (3) within 30 days of entering the treatment episode, and (4) proficient in English. Participants were excluded if they were: (1) prescribed opioid replacement therapy (e.g., buprenorphine, methadone), or (2) unable to provide informed consent. The study was approved by the Institutional Review Boards of the New York State Psychiatric Institute and all participating clinical sites. The study was registered on clinicaltrials.gov under the identifier NCT01104805. Additional details of program selection, design, and methods have been previously published (Campbell et al., 2012).
Design and Procedures
After providing a complete description of the study to each patient, written informed consent was obtained and a baseline assessment conducted. Following the assessment, participants were randomized to 12 weeks of either: (1) TAU; or (2) TAU + the computer-assisted, Internet-delivered TES, whereby TES was a substitute for approximately 2 hours of usual care (i.e., clinician-delivered groups). Randomization was stratified by: treatment site; patient’s primary substance of abuse (dichotomized as stimulant vs. non-stimulant); and whether or not the patient was abstinent at point of baseline assessment and study entry based on urine drug and breath alcohol tests. All participants were asked to provide self-reported substance use and urine drug and breath alcohol screens twice per week during the treatment phase; additional assessments were collected at weeks 4, 8, and 12.
Intervention
TAU in both intervention arms included site-specific recommended outpatient treatment attendance at each of the 10 treatment program sites. The recommended number of individual and group sessions per week differed across sites, but was on average 4–6 sessions.
The TES (Bickel et al., 2008) consists of 62 Internet-delivered, interactive, multimedia modules, grounded in the CRA (Budney & Higgins, 1998) and includes CM in the form of intermittent, escalating vouchers. An initial training module teaches participants how to use the program, followed by modules (or topics) on cognitive behavioral relapse prevention skills, psychosocial functioning, and HIV and other sexually transmitted infection prevention and treatment information. Video clips show actors modeling the skills being taught and short quizzes at the end of each module assess patient’s grasp of material and maximize individual mastery of the skills being taught. The CM component, also managed within the TES program, is a prize-based incentive system (Petry et al., 2005; Stitzer et al., 2010). Participants earn draws for submitting negative urine/breath alcohol screens and for completing TES modules (up to 4 per week). Draws are redeemed from a virtual “fish bowl” and yield congratulatory vouchers with messages (e.g., “good job”) or prizes of mostly modest value (usually around $1, occasionally around $20, rarely $80–$100).
Measures
Demographic and Clinical Characteristics
Sex, age, race/ethnicity, education, employment, insurance status, travel time/distance to program, internet use, and primary substance of abuse were assessed at baseline. Race/ethnicity was based on self-identification and categorized in the following way: White/non-Hispanic; Black or African American/non-Hispanic; Hispanic or Latino (White or Black); and multi-racial or other racial category (these subgroups were collapsed due to small numbers, see Table 1 for the frequency of each multi-racial and other subgroups). Primary substance of abuse was self-reported based on the substance with which the participant felt they had the largest problem or for which they were primarily seeking treatment. Abstinence at baseline/study entry was defined as negative results on both the urine drug and breath alcohol screens.
Table 1.
Racial identification among participants who endorsed multi-racial or another racial category other than White or Black/African American (n=72)
| Racial/Ethnic Identification | n (%) |
|---|---|
| Native Hawaiian-Pacific Islander (NH-PI) | 12 (16.7) |
| Asian | 11 (15.3) |
| White/AI-AN | 6 (8.3) |
| White/NH-PI | 5 (6.9) |
| African American/AI-AN | 7 (9.7) |
| Asian/NH-PI | 6 (8.3) |
| American Indian-Alaska Native (AI-AN) | 3 (4.2) |
| Other | 3 (4.2) |
| White/African American/AI-AN | 3 (4.2) |
| White/Asian/NH-PI | 3 (4.2) |
| White/AI-AN/Asian/NH-PI | 3 (4.2) |
| White/African American | 2 (2.8) |
| White/Asian | 1 (1.4) |
| African American/Other | 1 (1.4) |
| African American/NH-PI | 1 (1.4) |
| White/AI-AN/Other | 1 (1.4) |
| White/Asian/Other | 1 (1.4) |
| African American/NH-PI/Other | 1 (1.4) |
| Asian/NH-PI/Other | 1 (1.4) |
| White/African American/AI-AN/Other | 1 (1.4) |
Abstinence
Abstinence from drugs and alcohol was evaluated twice weekly during the 12-week treatment phase. Participant abstinence was based on: 1) a negative urine test for 10 drugs: cocaine, opioids, amphetamines, cannabinoids (THC), methamphetamines, benzodiazepines, oxycodone, methadone, barbiturates, and MDMA; and 2) self-reported abstinence from drugs and alcohol based on the Timeline Follow Back method (Sobell et al., 1992). A patient was considered to be abstinent if the urine screen and self-report were negative (for drugs and alcohol) and to be not abstinent otherwise. Data on abstinence was considered missing if the urine screen was missing or if the urine screen was negative and the self-report was missing. The outcome was a binary measure of abstinence (yes or no) during the last 4 weeks of treatment (i.e., weeks 9–12). Abstinence in the last four weeks of treatment was the pre-specified primary outcome in the study protocol since this is a time when the treatment effect was expected (Campbell et al., 2012) and shown to be constant (Campbell et al., 2014).
Retention
Retention was evaluated as a binary outcome (retained in treatment to week 12 versus dropped out before week 12). Retention data was collected from treatment program records and based on last face-to-face contact prior to discharge.
Social Functioning
Social functioning was measured using the 54-item Social Adjustment Scale Self-report (Weissman, 1999) which assesses instrumental (i.e., what we do in the world) and expressive (i.e., how we relate to others) role performance over the prior two weeks. It is comprised of questions covering six social roles (work [paid worker, student, or homemaker], social and leisure activities, extended family relationships, marital relationship, role as a parent, and role within the family unit), relevant to both males and females, and provides an overall indicator of social functioning (Weissman et al., 2001). Items are rated on a 5-point scale (1 to 5) with higher scores indicating greater social impairment.
Coping
Coping strategies were measured using the 23-item Coping Strategies Scale – Brief Version (CSS). The CSS is a self-report tool (adapted from the Process of Change Questionnaire, Prochaska et al., 1988) originally developed to assess change processes and skills used in modifying alcohol use behaviour (Litt et al., 2003). A shorter version of the tool was developed to evaluate the utilization of coping skills among marijuana users (Litt, Kadden, & Tennen, 2012). In this study, the wording was adapted to include all substances of abuse. The CSS evaluates the frequency of using different emotion- (e.g., think of difficulties as challenges) and problem-focused (e.g., deal with tension in other ways) coping strategies over a 12 week period of time, for which a score of 1 to 4 is obtained (1=never used, 2=seldomly used, 3=occasionally used, 4=frequently used). Each skill was then dichotomized as adopted (answer 3 or 4) or not adopted (answer 1 or 2) and the adopted strategies were summed to obtain a continuous score between 0 and 23. Calculating the CSS score in this manner is a clear and meaningful way to use the scale, as it provides an estimated number of coping strategies utilized by participants, and it has been used in a prior study (Sanchez et al., 2014).
Craving
Drug and alcohol craving was assessed by asking participants on how many days in the last seven they experienced an urge, desire or craving for drugs or alcohol and coded categorically: 0 days (did not experience any craving; n=219), 1–3 days (n=128), and 4–7 days (n=100). Social adjustment and craving were both measured at week 12.
Acceptability
Acceptability was comprised of both utility and satisfaction with TAU or TES using data collected across five indicators (0–10 point scales) at weeks 4, 8, and 12. Participants were asked in general (i.e., not for a specific timeframe) how useful (not at all to very), how much new information (none to a great deal), how easy to understand (very easy to very difficult; reverse coded), how interesting (not at all to very), and how satisfied (not at all to very) they were with TAU or with TES (Internet-delivered CRA modules and computer-assisted CM). Higher scores indicated a more positive perception, or greater acceptability, of the intervention. Similar indicators were used in a previous pilot study of TES with adolescents (Marsch et al., 2011).
Statistical Methods
Demographic and clinical characteristics were described using means, standard deviations, and frequencies as a function of each of the racial/ethnic subgroups (White, Black, Hispanic/Latino, multi-racial/other; 1 case was missing race/ethnicity). Chi-square tests were used to test differences between racial/ethnic subgroups (White, Black, Hispanic/Latino) on treatment process variables.
To explore differences in outcome (i.e., abstinence during the last four weeks of treatment, retention, social functioning, coping, and craving) by racial/ethnic subgroup, the following variables were included in generalized linear mixed effect models (with Proc GLIMMIX in SAS): treatment, race/ethnicity (White, Black, or Hispanic/Latino), abstinence at study entry, and baseline scores corresponding to the outcomes of social functioning, coping, and craving. The multi-racial/other subgroup was not included in these analyses given limited ability to interpret outcomes due to extreme diversity (see Table 1). Site and subject were treated as random effects except in the models testing social functioning and coping where site was treated as a fixed effect due to the models not converging because of inherent correlation between predictors and site. Interactions were tested (between treatment, race/ethnicity, and abstinence at study entry) and included in the final model if significant (p<.05). Time was included in the model testing abstinence (n=399; 35 cases removed that were missing all four weeks of data). The correlation between the repeated measurements within subject was modeled using the first-order auto regressive structure, and logit link function for the dichotomous outcome variable. The models for social functioning (normal distribution with identity link function) and craving (ordinal outcome with cumulative logit link function) included n=378 and n=376 cases, respectively (56 and 58 cases, respectively, missing both variables at week 12). Coping strategies (total score range in the sample = 0–23) was not normally distributed, therefore the score was reversed and logged to create a normal distribution (range = 0–3.18); identity link function was utilized (n=378; 56 cases removed that were missing week 12 coping data). Missing data was assumed missing at random.
Generalized linear mixed effect models were fit to explore the association between race/ethnicity and acceptability of TAU (n=398) and TES (n=200) at weeks 4, 8, and 12. The model exploring acceptability over time included race/ethnicity (White, Black, or Hispanic/Latino), abstinence at study entry, and time and in the case of TAU, intervention arm; interactions (race/ethnicity, abstinence at study entry, treatment) were tested and included if significant (55, 90, and 126 cases were missing TAU acceptability data at weeks 4, 8, and 12, respectively).
All hypothesis tests were considered two-sided and SAS version 9.3 was utilized for all analyses.
RESULTS
Baseline Demographic and Clinical Characteristics and Treatment Process by Race/Ethnicity
Table 2 presents baseline characteristics of the randomized sample by racial/ethnic category: White (n=267), Black or African-American (n=112), Hispanic/Latino (n=55), and multi-racial or other racial category (n=72). Black participants appeared to be older on average (M=38.1 years, SD=10.4) compared to Hispanics/Latinos (M=36.1 years, SD=11.3), multi-racial/other (M=35.1 years, SD=10.7) and Whites (M=33.2 years, SD=10.8). A greater percentage of Black and Hispanic/Latino participants did not have a high school diploma or GED (33% compared to 20% for Whites and 14% for multi-racial/other). Racial/ethnic minority subgroups also appeared less likely to be employed, have insurance (except for multi-racial/other), and access the Internet. 55% of Whites, compared to 37% and 45% of Blacks and Hispanics/Latinos respectively, reported accessing the Internet multiple times per day. Racial/ethnic minority subgroups reported more time for travel to the treatment program despite living the same distance away (approximately 7–9 minutes more time on average). White participants reported opioids (33%) most frequently as a primary substance of abuse, while Black participants reported cocaine (41%) and Hispanics/Latinos reported alcohol (29%) and cannabis (29%). Baseline levels of social functioning, coping, and craving appeared similar across racial/ethnic subgroups.
Table 2.
Baseline and Clinical Characteristics at Study Entry (N=506†)
| White (Non-Hispanic) n=267 |
Black (Non-Hispanic) n=112 |
Hispanic/Latino n=55 |
Multi-Racial/Other (Non-Hispanic) n=72 |
|
|---|---|---|---|---|
|
| ||||
| Mean (SD) or % | ||||
|
| ||||
| Sex (female) | 39.97 | 40.18 | 32.73 | 30.99 |
|
| ||||
| Age (years) | 33.21 (10.78) | 38.13 (10.44) | 36.13 (11.26) | 35.14 (10.70) |
|
| ||||
| Education | ||||
| < High School | 19.85 | 33.04 | 32.73 | 13.89 |
| High School/GED | 62.17 | 56.25 | 49.09 | 73.61 |
| > High School | 17.98 | 10.71 | 18.18 | 12.50 |
|
| ||||
| Currently Employed | 47.19 | 29.46 | 36.36 | 40.28 |
|
| ||||
| Insurance Status (yes) | 77.15 | 66.07 | 66.67 | 86.11 |
|
| ||||
| Distance from Program (miles) | 8.71 (9.07) | 8.57 (7.56) | 8.69 (6.19) | 10.18 (10.93) |
|
| ||||
| Travel time to Program (minutes) | 21.79 (17.55) | 30.68 (23.09) | 29.09 (24.65) | 31.32 (28.13) |
|
| ||||
| Internet Access (yes) | 82.02 | 58.93 | 69.09 | 69.44 |
|
| ||||
| Internet Use Frequency | ||||
| <1 day/wk | 26.59 | 46.43 | 30.91 | 40.28 |
| 1–6 days/wk | 18.73 | 16.96 | 23.64 | 19.44 |
| ≥daily | 54.68 | 36.61 | 45.45 | 40.28 |
|
| ||||
| Abstinent at Study Entry | 56.18 | 47.32 | 47.27 | 62.50 |
|
| ||||
| Primary Substance of Abuse | ||||
| Alcohol | 22.10 | 18.75 | 29.09 | 11.11 |
| Cannabis | 16.10 | 30.36 | 29.09 | 29.17 |
| Opioids | 33.33 | 5.36 | 14.55 | 5.56 |
| Cocaine | 15.36 | 41.07 | 14.55 | 9.72 |
| Amphetamines/Methamphetamines | 10.86 | 2.68 | 10.91 | 43.06 |
| Other | 2.25 | 1.79 | 1.82 | 1.39 |
|
| ||||
| Days of Substance Use (past 30 days) | ||||
| Alcohol | 3.79 (6.24) | 4.82 (7.33) | 3.71 (5.66) | 1.89 (4.42) |
| Cannabis | 4.61 (8.51) | 6.42 (9.92) | 4.95 (8.54) | 4.01 (7.50) |
| Opioids | 2.42 (5.08) | 0.29 (2.20) | 1.62 (5.21) | 0.35 (1.76) |
| Cocaine | 1.29 (3.41) | 2.61 (4.88) | 2.05 (5.29) | 0.69 (2.32) |
| Amphetamines/Meth | 0.83 (3.39) | 0.13 (0.93) | 0.78 (3.77) | 2.88 (6.58) |
|
| ||||
| Social functioning (total score) | 2.19 (0.46) | 2.19 (0.57) | 2.26 (0.49) | 2.06 (0.48) |
|
| ||||
| Coping Strategies (# endorsed; range=0–23) | 16.66 (5.62) | 16.91 (5.15) | 16.75 (5.27) | 17.25 (5.58) |
|
| ||||
| Craving (past 7 days) | ||||
| 0 days | 43.77 | 34.82 | 43.64 | 38.03 |
| 1–3 days | 28.30 | 33.93 | 23.64 | 29.58 |
| 4–7 days | 27.92 | 31.25 | 32.73 | 32.39 |
excluded n=1 “refused to answer” racial/ethnic category.
Table 3 shows treatment process outcomes as a function of TAU and TES by race/ethnicity. Among participants randomized to TAU, no significant differences were found by race/ethnicity (White, Black, and Hispanic/Latino) on the days attending TAU (χ2(2)=1.71, p=.425) or the number of TAU sessions attended (χ2(2)=2.77, p=.251). Among participants randomized to TES, no significant differences were found by race/ethnicity on the days attending TAU (χ2(2)=2.43, p=.296) or the number of TAU sessions attended (χ2(2)=3.08, p=.215). There were also no significant differences by race/ethnicity on the number of TES modules completed (χ2(2)=3.14, p=.208); all subgroups completed 35 modules or more (range = 34.91–39.21) out of a recommended 48 (4 per week over 12 weeks) (module completion was tracked automatically via the TES program).
Table 3.
Treatment process outcomes by treatment arm (treatment-as-usual [TAU] and Therapeutic Education System [TES]) and race/ethnicity subgroup (n=434)
| White (Non-Hispanic) | Black (Non-Hispanic) | Hispanic/Latino | Test Statistic, p-value | |
|---|---|---|---|---|
|
| ||||
| M (SD) | ||||
|
|
||||
| TAU Arm | n=141 | n=44 | n=29 | |
|
|
||||
| TAU Days Attended (total)a | 14.96 (11.15) | 12.20 (8.90) | 15.58 (12.19) | X2(2)=1.71, p=.425 |
| TAU Sessions Attended (total)a | 19.50 (15.36) | 14.43 (10.58) | 23.59(22.50) | X2(2)=2.77, p=.251 |
|
|
||||
| TES Arm | n=126 | n=68 | n=26 | |
|
|
||||
| TAU Days Attended (total)a | 15.14 (11.75) | 15.00 (9.86) | 18.00 (9.27) | X2(2)=2.43, p=.296 |
| TAU Sessions Attended (total)a | 19.10 (16.55) | 17.19 (11.74) | 24.46 (16.72) | X2(2)=3.08, p=.215 |
| TES Modules Completed (total)b | 34.91 (17.51) | 39.16 (16.66) | 36.81(15.34) | X2(2)=3.14, p=.208 |
84 days (12 weeks) of treatment phase;
number of modules recommended for completion during 12 weeks of treatment = 48;
Non-parametric Kruskal-Wallis tests were used.
Race/Ethnicity as a Moderator of Treatment on Primary and Secondary Outcomes
Table 4 presents the final models for each of five outcomes: abstinence, retention, social functioning, coping, and craving. For all outcomes, the interaction between treatment, race/ethnicity, and abstinence at study entry were tested and included in the final model if significant. For the abstinence model, time was also tested for interaction with these variables.
Table 4.
Final GLM models exploring race/ethnicity (White, Black/African American, Hispanic/Latino) as a moderator of the intervention (TES v TAU) effect on (A) abstinence in last four weeks of treatment, (B) retention at week 12 (yes/no), (C) social functioning at week 12, (D) coping skills at week 12, and (E) craving in the last 7 days of treatment. The three-way and two-way interactions between treatment, abstinence at study entry (positive or negative), and the race/ethnicity moderator were tested and included in the final model if significant (p<.05). Time was included in the abstinence model (A) to test the multiple data points in the last four weeks of treatment. Baseline social functioning, coping, and craving were controlled in Models C-E, respectively. Site was included as a random effect in Models A, B, E; site was included as a fixed effect in Models C and D.
| F-statistic | p-value | |
|---|---|---|
| A: Abstinence, Final Four Weeks of Treatment (n=399) | ||
| Race/Ethnicity (White, Black, Hispanic) | F(2,2081)=0.26 | .776 |
| Abstinent at Study Entry | F(1,2081)=49.27 | <.001 |
| Time x Treatment (TES vs TAU) | F(1,2081)=4.41 | .036 |
|
| ||
| B: Retention at Week 12 (n=434) | ||
| Race/Ethnicity (White, Black, Hispanic) | F(2, 420)=0.19 | .827 |
| Treatment (TES vs TAU) | F(1, 420)=2.74 | .099 |
| Abstinent at Study Entry | F(1, 420)=4.47 | .035 |
|
| ||
| C: Social Functioning at Week 12, total score (n=378) | ||
| Race/Ethnicity (White, Black, Hispanic) | F(2,363)=2.80 | .062 |
| Abstinent at Study Entry | F(1,363)=0.04 | .846 |
| Treatment (TES vs TAU) | F(1,363)=1.24 | .266 |
| Baseline Social Functioning | F(1,363)=98.45 | <.001 |
| Site | F(9,363)=0.70 | .713 |
|
| ||
| D: Coping Strategies at Week 12, mean score (n=378) | ||
| Treatment (TES vs TAU) × Race (White, Black, Hispanic) × Abstinent at Study Entry | F(2,356)=3.62 | .028 |
| (Log) Baseline Coping | F(2,356)=78.42 | <.001 |
| Site | F(9,356)=1.02 | .421 |
|
| ||
| E: Craving at Week 12, last 7 days (n=376) | ||
| Race/Ethnicity (White, Black, Hispanic) | F(2,359)=0.57 | .565 |
| Baseline Craving | F(1,359)=40.61 | <.001 |
| Treatment (TES v TAU) × Abstinent at Study Entry | F(1,359)=6.01 | .015 |
Abstinence (Table 4A)
In the final abstinence model, there was one significant two-way interaction between time (week) and treatment (p=.04) favoring TES over TAU although differences decreased over time (b=1.33, t(2081)=3.46, p=.001 at half-week 17; b=0.67, t(2081)=1.76, p=.08, at half-week 24). Race/ethnicity did not moderate the treatment effect (p=.89) and there was no main effect of race/ethnicity on abstinence (p=.776). The 2-way interaction of time and race/ethnicity approached, but did not reach, significance (p=.051). Abstinence at study entry was significantly associated with abstinence in the final four weeks of treatment (p<.001).
Retention (Table 4B)
In the final retention model, there was a significant effect of abstinence at study entry (p=.035); those who were baseline negative were more likely to be retained in treatment at week 12. The association between treatment and retention did not reach the significance cut-off (p=.10). Race/ethnicity was not a significant moderator of treatment (p=.19) and there was no main effect of race/ethnicity on retention (p=.827).
Social Functioning (Table 4C)
There were no three-way or two-way significant interactions between treatment, abstinence at study entry, and race/ethnicity on the outcome of social functioning at week 12 as measured by the Social Adjustment Scale. The main effect of race/ethnicity approached, but did not reach, significance (p=.062). Baseline social functioning was significantly associated with week 12 social functioning (p<.001). Treatment and abstinence at study entry were not significantly associated with social functioning (p=.266 and p=.846, respectively). Site was included as a fixed effect in this model; it was not significantly associated with social functioning (p=.713).
Coping Strategies (Table 4D and Figure 1)
Figure 1.
Observed geometric means of coping skills (y-axis) by race/ethnicity (x-axis) at the end of treatment split by abstinence at study entry (data exhibit right skewed behavior and were analyzed using log-normal distribution; thus using geometric means to portray the typical behavior is the most appropriate)
In the model assessing mean coping skills at week 12, there was a significant three-way interaction between treatment, race/ethnicity, and abstinence at study entry (p=.028). As portrayed in Figure 1, among White participants who were not abstinent at study entry (baseline positive, Figure 1b), TES was significantly associated with greater coping skills at end of treatment compared to TAU (p=.008); among Black participants abstinent at study entry (baseline negative, Figure 1a), TES was significantly associated with greater coping scores at end of treatment compared to TAU participants (p<.001). Baseline coping skills were significantly associated with week 12 coping skills (p<.001). Site was included as a fixed effect in this model; it was not significantly associated with coping skills (p=.421).
Craving (Table 4E)
There was one significant two-way interaction between treatment and abstinence at study entry (p=.015). Among those not abstinent at baseline (baseline positive), TES participants were more likely to be in a lower days of craving category compared to TAU participants (p=013). There was no significant difference between TES and TAU among participants who were abstinent at study entry (baseline negative) (p=.351). Race/ethnicity was not significantly associated with craving at week 12 (p=.565). Baseline craving was significantly associated with week 12 craving (p<.001).
Acceptability
At week 12, raw TAU acceptability scores ranged from a mean of 7.55 (SD=2.01) for White participants to 8.05 (SD=1.72) for Black participants on a 0–10 point scale (Hispanic/Latino participants’ acceptability was M=7.78 (SD=1.72)). Table 5 presents the final models exploring the association between race/ethnicity and acceptability of TAU and TES over time (weeks 4, 8, and 12) controlling for abstinence at study entry. All participants attended TAU at the participating treatment programs. There was a significant main effect of race/ethnicity (p=.003) on acceptability of TAU (Table 5A). Compared to Hispanic/Latino (b=0.6761, t(632)=2.66, p=.008) and Black (b=0.6310, t(632)=2.77, p=.006) participants, White participants reported significantly lower acceptability of TAU. Among participants randomized to TES, acceptability of TAU was higher, although this did not reach the significance cut-off (p=.054). Abstinence at study entry was not significantly associated with acceptability of TAU (p=.091).
Table 5.
Final GLM models exploring the association between race/ethnicity (White, Black/African American, Hispanic/Latino) and acceptability of (A) treatment-as-usual (TAU) and (B) Therapeutic Education System (TES) over time (weeks 4, 8, and 12), controlling for abstinence status at study entry (positive or negative).
| F-statistic | p-value | |
|---|---|---|
| A: Acceptability of TAU (n=398) | ||
| Time (continuous, 4, 8, 12) | F(1,632)=0.15 | .698 |
| Race/Ethnicity (White, Black, Hispanic) | F(2,632)=5.85 | .003 |
| Abstinent at Study Entry | F(1,632)=2.87 | .091 |
| Treatment (TES vs TAU) | F(1,632)=3.72 | .054 |
|
| ||
| B: Acceptability of TES (n=200 TES arm only) | ||
| Time (continuous variable, 4, 8, 12) | F(1, 340)=0.02 | .878 |
| Race/Ethnicity (White, Black, Hispanic)) | F(2, 340)=3.75 | .025 |
| Abstinent at Study Entry | F(1, 340)=0.96 | .327 |
At week 12, raw TES acceptability scores ranged from a mean of 8.05 (SD=1.64) for Hispanic/Latino participants to 8.66 (SD=1.31) for Black participants on a 0–10 point scale (White participants’ acceptability was M=8.08 (SD=1.60)). In the model examining acceptability of TES (for those randomized to TES, n=200), there was a significant main effect of race/ethnicity (p=.025) (Table 5B). White participants reported significantly lower TES acceptability scores compared to Black participants (b=.687, t(340)=2.71, p=.007); however there was no significant difference between White and Hispanic/Latino participants (b=0.105, t(340)=0.33, p=.743). Abstinence at study entry was not significantly associated with acceptability of TES (p=.327).
DISCUSSION
This study is unique in the analysis of race/ethnicity as a moderator of treatment outcomes from an Internet-delivered psychosocial intervention for substance use disorders. Overall findings demonstrate that among a diverse sample of individuals in outpatient substance abuse treatment, there were no major differences in the effect of the Therapeutic Education System (TES); all racial/ethnic groups appeared to benefit. Black and Hispanic/Latino participants experienced similar outcomes compared to White participants across a range of clinical measures, including abstinence at the end of 12 weeks of study treatment, retention in treatment, social functioning, and days of craving.
There were mixed findings for the coping skills outcome. Among participants in TES, greater coping skills were reported by White participants who were not abstinent at study entry (baseline positive) and for Black participants who were abstinent at study entry (baseline negative). Coping skills have been shown to be a significant predictor of treatment outcomes (Dolan, Rohsenow, Martin & Monti, 2013; Litt, Kadden, Cooney, & Kabela, 2003). Reasons most often endorsed for using drugs and alcohol include relieving depression, achieving or maintaining euphoria, and improving self-confidence and social abilities (Bizzarri et al., 2007). There is empirical support for development of effective coping skills through computer-based interventions (Chaple et al., 2014; Kiluk & Carroll, 2013; Kiluk, Nich, Babuscio & Carroll, 2010) and findings from the current study suggest various subgroups may require tailored interventions for improving their coping skills. Tailoring of coping skills may also enhance relevance and acceptability regardless of whether a person is actively using drugs and alcohol. Cultural differences in coping strategies might also influence how individuals respond to the skills taught in TES (e.g., religious coping among Black individuals) (Ward, Clark, & Heidrich, 2009). Additional research is needed to disentangle this finding, including measures of coping strategies that are culturally specific or inclusive and in ways in which active substance use may impede or facilitate engagement with the content or with being able to incorporate the skill.
Overall, White participants reported lower acceptability compared to Black participants for both TAU and TES. White participants reported lower acceptability compared to Hispanic/Latino participants on TES only. TES was developed to be self-directed, teaching general skill development related to CRA principles. It could be that the higher acceptability among Black and Hispanic/Latino groups was a function of being able to individualize and have greater autonomy over their treatment (e.g., faster or slower pace, using personal experiences as examples). Future research including qualitative data collection could assist in identifying more specific aspects of technology-based treatment which patients find more or less attractive. It should be noted, however, that overall acceptability of TES was high across racial/ethnic subgroups with Whites and Hispanics/Latinos reporting a mean score of 8.1 and Black participants a mean score of 8.7 (range=0–10) at the end of treatment.
Black and Hispanic/Latino participants appear to have less accessibility to and potentially fewer resources for attaining health services as demonstrated by lower employment rates, lower rates of insurance, and greater time to travel to the treatment program (despite living the same distance from the program). In addition, age differences at treatment entry among racial/ethnic subgroups corresponds with previous research demonstrating that minority populations were least likely to initiate substance abuse treatment resulting in potentially longer periods of substance abuse or more severe substance use disorders (Acevedo et al., 2012).
Internet access overall, and frequency of use, was less among Black and Hispanic/Latino participants. Internet access is influenced by age, education, and socioeconomic status (Perrin & Duggan, 2015) and this is reflected in the demographic characteristics of the study sample; Black and Hispanic/Latino participants have an older mean age, lower high school graduation rates, and lower employment. Acceptability of TES, however, was just as high or higher among Black and Hispanic/Latino compared to White participants. With internet access increasing across all populations, technology-based interventions may be a useful way to address structural barriers to addiction treatment among racial/ethnic minority clients. For example, according to recent data, the gap in Internet access between Blacks, Hispanics/Latinos and Whites has narrowed; currently 78% of Blacks and 81% of Hispanics/Latinos use the internet compared to 85% of Whites (Perrin & Duggan, 2015). Further, racial/ethnic minorities are more likely to rely on Internet access exclusively through smartphones (12% of Blacks and 13% of Hispanics/Latinos versus only 4% of Whites) (Smith, 2015). Thus, utilizing “apps” to deliver addiction interventions through smartphones will clearly be an important treatment tool moving forward.
Limitations
Although this study benefits from a number of strengths, including a large sample size and geographic diversity, several limitations should be noted. First, this was a secondary analysis not originally powered to detect differences by racial/ethnic subgroups. Exploring race/ethnicity as a moderator of treatment was described a priori in the protocol, but sample size cannot be ruled out as the reason for not detecting racial/ethnic subgroup differences. Second, acceptability was assessed using five indicators; however the measure did not distinguish between the computer modality of the TES intervention and the treatment components. Further, there may have been other factors not assessed that could explain differences in acceptability between White, Black, and Hispanic/Latino participants. In addition, this study did not examine measurement equivalence of the questionnaire items when comparing responses across racial/ethnic subgroups. Research has demonstrated that cultural norms and practices often influence individuals’ perceived relevance and interpretation of items on assessment measures (Burlew, Weekes, Montgomery, et al., 2011). Finally, exploring outcomes among racial/ethnic subgroups outside of White, Black, and Hispanic/Latino was prevented due to small sample sizes. Conducting research on technology-based interventions specifically with American Indians/Alaska Natives, Asians, and Native Hawaiians/Pacific Islanders is critical to providing quality treatment and reducing health disparities.
Conclusion
Findings from this study lend additional support for the use of technology-based interventions in the treatment of substance use disorders. The acceptability of Internet-delivered interventions among racial/ethnic minority populations suggests promise for increasing access to services and reducing disparities in treatment outcomes. In this large multi-site national study, racial/ethnic subgroups received similar benefit from Internet-based CRA/CM and reported high rates of acceptability, with Black participants reporting the highest rates of acceptability. TES should be considered as an additional tool to support usual care in outpatient treatment programs among diverse subgroups of patients. Future research should explore tailoring of coping skills training to enhance engagement and acquisition among patients of varying race/ethnicities and substance use disorder severity.
Acknowledgments
This work was supported by grants from the National Drug Abuse Treatment Clinical Trials Network (CTN), NIDA: U10 DA013035 (Edward V. Nunes and John Rotrosen), U10 DA015831 (Kathleen M. Carroll and Roger D. Weiss), U10 DA013034 (Maxine L. Stitzer and Robert P. Schwartz), U10 DA013720 (José Szapocznik and Lisa R. Metsch), U10 DA013732 (Theresa Winhusen), U10 DA020024 (Madhukar H. Trivedi), U10 DA013714 (Dennis M. Donovan and John Roll), U10 DA015815 (James L. Sorensen and Dennis McCarty), U10 DA013045 (Walter Ling), and K24 DA022412 (Edward V. Nunes).
The authors wish to acknowledge the role of the research and clinical staff at the 10 recruitment sites, and gratitude for the time committed by study participants.
Footnotes
Disclosures
Dr. Nunes has received medication for research studies from Alkermes/Cephalon, Duramed Pharmaceuticals, and Reckitt-Benckiser. The other authors report no financial relationships with commercial interest.
References
- Acevedo A, Garnick DW, Lee MT, Horgan CM, Ritter G, Panas L, et al. Racial and ethnic differences in substance abuse treatment initiation and engagement. Journal of ethnicity in substance abuse. 2012;11(1):1–21. doi: 10.1080/15332640.2012.652516. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bewick BM, Trusler K, Barkham M, Hill AJ, Cahill J, Mulhern B. The effectiveness of web-based interventions designed to decrease alcohol consumption - A systematic review. Preventitive Medicine. 2008;47(1):17–26. doi: 10.1016/j.ypmed.2008.01.005. [DOI] [PubMed] [Google Scholar]
- Bickel WK, Marsch LA, Buchhalter A, Badger G. Computerized behavior therapy for opioid dependent outpatients: A randomized, controlled trial. Experimental Clinical Psychopharmacology. 2008;16:132–143. doi: 10.1037/1064-1297.16.2.132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bizzarri JV, Rucci P, Sbrana A, Gonnelli C, Massei GJ, Ravani L, et al. Reasons for substance use and vulnerability factors in patients with substance use disorder and anxiety or mood disorders. Addictive Behaviors. 2007;32(2):384–391. doi: 10.1016/j.addbeh.2006.04.005. [DOI] [PubMed] [Google Scholar]
- Budney AJ, Higgins ST. Therapy manuals for drug addiction, a community reinforcement plus vouchers approach: treating cocaine addiction. Rockville, MD: National Institute on Drug Abuse; 1998. [Google Scholar]
- Burlew AK, Weekes J, Montgomery L, Feaster DJ, Robbins MS, Rosa CL, Ruglass LM, Venner KL, Wu L. Conducting research with ethnic minorities: Methodological lessons from the NIDA Clinical Trials Network. The American Journal of Drug and Alcohol Abuse. 2011;37(5):324–332. doi: 10.3109/00952990.2011.596973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Campbell ANC, Nunes EV, Matthews AG, Stitzer M, Miele GM, Polsky D, … Ghitza UE. Internet-delivered treatment for substance abuse: A multisite randomized controlled trial. American Journal of Psychiatry. 2014;171(6):683–690. doi: 10.1176/appi.ajp.2014.13081055. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Campbell ANC, Nunes EV, Miele GM, Matthews A, Polsky D, Ghitza UE, … Crowell AR. Design and methodological considerations of an effectiveness trial of a computer-assisted intervention: An example from the NIDA Clinical Trials Network. Contemporary Clinical Trials. 2012;33(2):386–295. doi: 10.1016/j.cct.2011.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carroll KM, Rounsaville BJ. Computer-assisted therapy in psychiatry: Be brave-it’s a new world. Current Psychiatry Reports. 2010;12(5):426–432. doi: 10.1007/s11920-010-0146-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chaple M, Sacks S, McKendrick K, Marsch LA, Belenko S, Leukefeld C, et al. Feasibility of a computerized intervention for offenders with substance use disorders: a research note. Journal of Experimental Criminology. 2014;10(1):105–127. doi: 10.1007/s11292-013-9187-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dallery J, Gleen IM, Raiff BR. An internet-based abstinences reinforcement treatment for cigarette smoking. Drug and Alcohol Dependence. 2007;86:230–238. doi: 10.1016/j.drugalcdep.2006.06.013. [DOI] [PubMed] [Google Scholar]
- Dolan SL, Rohsenow DJ, Martin RA, Monti PM. Urge-specific and lifestyle coping strategies of alcoholics: Relationships of specific strategies to treatment outcome. Drug and Alcohol Dependence. 2013;128(1–2):8–14. doi: 10.1016/j.drugalcdep.2012.07.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Garcia-Fernandez G, Secades R, Garcia-Rodrigquez O, Sanchez-Hervas E, Fernandez-Hermida JR, Higgins ST. Adding voucher-based incentives to community reinforcement approach improves outcomes during treatment for cocaine dependence. The American Journal on Addictions. 2011;20(5):456–461. doi: 10.1111/j.1521-0391.2011.00154.x. [DOI] [PubMed] [Google Scholar]
- Gibbons MC. Use of health information technology among racial and ethnic underserved communities. Perspectives in Health Information Management. 2011;8:1f. [PMC free article] [PubMed] [Google Scholar]
- Godley S, Hedges K, Hunter B. Gender and racial differences in treatment process and outcome among participants in the adolescent community reinforcement approach. Psychology of Addictive Behaviors. 2011;25:143–154. doi: 10.1037/a0022179. [DOI] [PubMed] [Google Scholar]
- Higgins ST, Sigmon SC, Wong CJ, Heil SH, Badger GJ, Donham R, Dantona RL, Anthony S. Community reinforcement therapy for cocaine-dependent outpatients. Archives of General Psychiatry. 2003;60:1043–1052. doi: 10.1001/archpsyc.60.9.1043. [DOI] [PubMed] [Google Scholar]
- Kiluk BD, Carroll KM. New Developments in Behavioral Treatments for Substance Use Disorders. Current Psychiatry Reports. 2013;15(12):9. doi: 10.1007/s11920-013-0420-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kiluk BD, Nich C, Babuscio T, Carroll KM. Quality versus quantity: acquisition of coping skills following computerized cognitive-behavioral therapy for substance use disorders. Addiction. 2010;105(12):2120–2127. doi: 10.1111/j.1360-0443.2010.03076.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Litt MD, Kadden RM, Cooney NL, Kabela E. Coping skills and treatment outcomes in cognitive-behavioral and interactional group therapy for alcoholism. Journal of Consulting & Clinical Psychology. 2003;71(1):118–28. doi: 10.1037//0022-006x.71.1.118. [DOI] [PubMed] [Google Scholar]
- Litt MD, Kadden RM, Tennen H. The nature of coping in treatment for marijuana dependence: latent structure and validation of the Coping Strategies Scale. Psychology of Addictive Behaviors: Journal of the Society of Psychologists in Addictive Behaviors. 2012;26(4):791–800. doi: 10.1037/a0026207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marsch LA, Grabinski MJ, Bickel WK, Desrosiers A, Guarino H, Muehlbach B, Acosta M. Computer-assisted HIV prevention for youth with substance use disorders. Substance Use and Misuse. 2011;46(1):46–56. doi: 10.3109/10826084.2011.521088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Montgomery L, Carroll K, Petry NM. Initial abstinence status and contingency management treatment outcomes among adults: Does race matter? Journal of Consulting and Clinical Psychology. doi: 10.1037/a0039021. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moore BA, Fazzino T, Garnet B, Cutter CJ, Barry DT. Computer-based interventions for drug use disorders: A systematic review. Journal of Substance Abuse Treatment. 2011;40(3):215–223. doi: 10.1016/j.jsat.2010.11.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Institue on Drug Abuse. DrugFacts: Treatment statistics. 2011 Mar; Retrieved from: http://www.drugabuse.gov/publications/drugfacts/treatment-statistics.
- Perrin A, Duggan M. Americans’ internet access: 2000–2015. Pew Research Center; 2015. Jun 26, Retrieved from: http://www.pewinternet.org/2015/06/26/americans-internet-access-2000-2015/ [Google Scholar]
- Petry NM, Peirce JM, Stitzer ML, Blaine J, Roll JM, Cohen A, … Li R. Effect of prize-based incentives on outcomes in stimulant abusers in outpatient psychosocial treatment programs: A National Drug Abuse Treatment Clinical Trials Network study. Archives of General Psychiatry. 2005;62:1148–1156. doi: 10.1001/archpsyc.62.10.1148. [DOI] [PubMed] [Google Scholar]
- Prochaska JO, Velicer WF, DiClemente CC, Fava J. Measuring processes of change: Applications to the cessation of smoking. Journal of Consulting & Clinical Psychology. 1988;56(4):520–8. doi: 10.1037//0022-006x.56.4.520. [DOI] [PubMed] [Google Scholar]
- Rapp RC, Xu J, Carr CA, Lane DT, Wang J, Carlson R. Treatment barriers identified by substance abusers assessed at a centralized intake unit. Journal of Substance Abuse Treatment. 2006;30:227–235. doi: 10.1016/j.jsat.2006.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reynolds B, Dallery J, Shroff P, Patak M, Leraras K. A web-based contingency management program with adolescent smokers. Journal of Applied Behavioral Analysis. 2008;41(4):597–601. doi: 10.1901/jaba.2008.41-597. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Saloner B, Le Cook B. Blacks and Hispanics are less likely than Whites to complete addiction treatment, largely due to socioeconomic factors. Health Affairs. 2013;32(1):135–145. doi: 10.1377/hlthaff.2011.0983. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sanchez K, Walker R, Campbell ANC, Greer TL, Hu M, Grannemann BD, Nunes EV, Trivedi MH. Depressive symptoms and associated clinical characteristics in outpatients seeking community-based treatment for alcohol and drug problems. Substance Abuse. 2015;36(3):297–303. doi: 10.1080/08897077.2014.937845. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith A. Mobile access 2010. Pew Research Center; 2010. Jul 7, Retrieved from http://www.pewinternet.org/2010/07/07/mobile-access-2010/ [Google Scholar]
- Smith A. U.S. smartphone use in 2015. Pew Research Center; 2015. Apr 1, Retrieved from: http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/ [Google Scholar]
- Smith JE, Meyers RJ, Miller WR. The community reinforcement approach to the treatment of substance use disorders. American Journal of Addictions. 2001;10:S51–9. doi: 10.1080/10550490150504137. [DOI] [PubMed] [Google Scholar]
- Sobell M, Sobell L, Bogardis J, Leo G, Skinner W. Problem drinkers’ perceptions of whether treatment goals should be self-selected or therapist-selected. Behavioral Therapist. 1992;23:43–52. [Google Scholar]
- Stitzer ML, Petry NM, Peirce JM. Motivational incentives research in the National Drug Abuse Treatment Clinical Trials Network. Journal of Substance Abuse Treatment. 2010;38(S1):S61–69. doi: 10.1016/j.jsat.2009.12.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vahabzadeh M, Lin J, Mezghanni M, Epstein DH, Preston KL. Automation in an addiction treatment research clinic. Computerised contingency management, ecological momentary assessment and a protocol workflow system. Drug and Alcohol Review. 2009;28(1):3–11. doi: 10.1111/j.1465-3362.2008.00007.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ward EC, Clark le O, Heidrich S. African American women’s beliefs, coping behaviors, and barriers to seeking mental health services. Qualitative Health Research. 2009;19(11):1589–1601. doi: 10.1177/1049732309350686. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weissman MM. Social Adjustment Scale – Self Report. North Tonawanda, NY: Multi Health Systems; 1999. [Google Scholar]
- Weisman MM, Olfson M, Gameroff MJ, Feder A, Fuentes M. A comparison of three scales of assessing social functioning in primary care. American Journal of Psychiatry. 2001;158:460–466. doi: 10.1176/appi.ajp.158.3.460. [DOI] [PubMed] [Google Scholar]
- Wood SK, Eckley L, Hughes K, Hardcastle KA, Bellis MA, Schrooten J, … Voorham L. Computer-based programmes for the prevention and management of illicit recreational drug use: A systematic review. Addictive Behaviors. 2014;39(1):30–38. doi: 10.1016/j.addbeh.2013.09.010. [DOI] [PubMed] [Google Scholar]