Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 May 1.
Published in final edited form as: J Affect Disord. 2018 Feb 17;232:219–228. doi: 10.1016/j.jad.2018.02.053

A randomized controlled trial of an Internet delivered dialectical behavior therapy skills training for suicidal and heavy episodic drinkers

Chelsey R Wilks a,✉, Anita Lungu b, Sin Yee Ang a, Brandon Matsumiya c, Qingqing Yin a, Marsha M Linehan a
PMCID: PMC5859943  NIHMSID: NIHMS947555  PMID: 29499504

Abstract

Background

Given that alcohol misuse elevates risk of suicide death among ideators, the paucity of treatment outcome research for individuals presenting with both suicide ideation and problem drinking is particularly troubling. Dialectical Behavior Therapy (DBT) skills training, which effectively targets behaviors associated with emotion dysregulation including addictive and suicidal behaviors, provides a fitting model amenable to computerization. As stigma and scarcity stand as potential barriers to treatment, online dissemination platforms provide means for efficient treatment delivery that can augment the utility of suitable interventions. This pilot RCT sought to evaluate the feasibility, acceptability, and preliminary efficacy of an Internet-delivered DBT skills training intervention (iDBT-ST) for suicidal individuals who engage in heavy episodic drinking

Methods

Participants (N = 59) were randomized to receive iDBT-ST immediately or after an 8-week waiting period. Clinical outcomes were suicide ideation, alcohol use, and emotion dysregulation.

Results

Participants on average saw a significant reduction in all outcomes over the four-month study period. Compared to waitlist controls, individuals who received iDBT-ST immediately showed faster reductions in alcohol consumption.

Conclusions

Preliminary results suggest that iDBT-ST may be a viable resource for the high-risk and underserved group represented in this study, and pathways for future development are suggested.

Limitations

There was difficulty retaining and engaging participants due to technological barriers.

Keywords: Dialectical behavior therapy, Alcohol, Suicide, Randomized controlled trial, Computerized psychotherapy, Internet

Suicide is a significant public health problem. In the United States, suicide is the tenth leading cause of death [Centers for Disease Control and Prevention (CDC), 2016]. Chronic alcohol use and abuse has been identified as a uniquely salient risk factor for suicide among suicide ideators, making ideators who also meet criteria for an alcohol use disorder (AUD) among one of the most at-risk populations for suicide (Han, et al., 2015; Kelly, et al., 2001; McManama et al. 2014; Nock et al., 2010). In particular, approximately 40% of suicides are preceded by acute alcohol intoxication (Cherpitel, Borges, & Wilcox, 2004). Among individuals physically dependent on alcohol, risk for suicide is 60–120 times greater than it is for non-clinical populations (Murphy and Wetzel, 1990), and simply meeting criteria for an AUD increases one’s lifetime risk for suicide to 7% (Brady, 2005). Taken together, alcohol-involved suicide is a considerable and complex problem.

While the relationship between alcohol and suicide remains a complex one, research has consistently pointed to emotion dysregulation as a common link among those who engage in both heavy drinking and suicidal behavior (Conner et al., 2009; Nordström, et al., 1995; Brown, et al., 2002). Heavy episodic drinking (HED), defined as consuming four (for women) or five (for men) standard alcoholic beverages in a two-hour period, is particularly associated with both suicide and emotion dysregulation (Bagge et al. 2013; Kaplan et al. 2014; Kuvaas et al., 2014). Thus, suicide ideation and alcohol misuse, particularly HED, are theorized to function as problem solving behaviors employed in lieu of effective strategies to cope with intense negative emotions (Linehan, 1993a; Schneidman 1993; Slee et al., 2008a; Slee et al., 2008b).

Given the problem of alcohol-involved suicides, there remains a significant treatment gap for individuals presenting with high-risk behaviors (Bruffaerts et al., 2011; Patel et al., 2016). Web-based or computerized interventions (CI), which incorporate software that automates many aspects of psychotherapy (e.g. computerized cognitive behavioral therapy cCBT; Kaltenthaler et al., 2006), have been deployed as one way to bridge the treatment gap. In published RCTs of cCBT that target mood or anxiety disorders, individuals presenting with suicide ideation and/or co-occurring substance use, are yet to be targeted by CI. Specifically, one meta-analysis found that over 90% of RCTs of cCBT excluded individuals at risk for suicide, and 74% excluded individuals engaging in addictive behavior (Wilks, Zieve, & Lessing, 2016). This trend is contrasted by emerging evidence that suicidal individuals may be more likely to seek out help online compared to face-to-face consultations (Wilks, Coyle, Krek, Lungu, & Andriani 2017). Promisingly, a growing base of research evidence suggests that Internet-delivered interventions are effective in reducing problematic drinking (Rooke, Thorsteinsson, Karpin, Copeland, & Allsop, 2010; White et al., 2010), as well as suicide ideation (Lai, Maniam, Chan, & Ravindran, 2014.). However, there remains a clear gap in the literature for treatment outcome research on individuals presenting with suicide ideation and problematic drinking, and on web-based interventions for individuals presenting with complex and high-risk behaviors.

Dialectical behavior therapy (DBT; Linehan, 1993; Linehan 2014) is a behavioral treatment specifically designed for individuals at high risk for suicide who present with co-occurring behavioral dysfunction related to emotion dysregulation. In addition, the skills training component of DBT has been shown to be effective in reducing behaviors associated with emotion dysregulation in numerous RCTs (Linehan, 2014; Valentine et al., 2014), suicidal behavior (Linehan et al., 2015), and problematic alcohol use (Whiteside, 2011). Moreover, much like CBT, DBT is amendable to computerization given its logical flow and modular structure (Linehan & Wilks, 2015; Lungu, 2015). This study thus aims to preliminarily evaluate an Internet-delivered DBT skills training intervention against a waitlist control on a sample of suicidal individuals who engage in HED and experience emotion regulation difficulties.

Method

Study Design

This study is a pilot RCT comparing individuals who received iDBT-ST immediately to those on an 8-week waiting list. Using a 1:1 allocation ratio, participants were randomized to receive iDBT-ST immediately (n=30) or to a waitlist (n=29). A minimization randomization procedure (White & Freedman, 1978) was used to match participants on three primary prognostic variables: (1) biological sex assigned at birth, (2) severity of disordered drinking, and (3) severity of suicide ideation. Participants were assessed at pre-treatment, and at monthly intervals for four months. All assessments were conducted online using Qualitrics.com. The periods of enrollment and follow-up ran from June 2016 through September 2016. Figure 1 summarizes the flow of participants through the study. All study procedures were conducted in accordance with IRB approved procedures.

Figure 1.

Figure 1

Open in a new tab

Consort Diagram

Participants

Participants (N=59) were recruited via online forums accessible throughout the United States and consecutively enrolled into the study. Inclusion criteria were: (1) endorsed the presence of suicide ideation in the past month (defined as answering >1 on Item 1 of the Suicide Behavior Questionnaire-Revised; Osman et al. 2001), (2) endorsed two or more episodes of HED (defined as reporting consumption of at least 4 standard drinks for women and 5 drinks for men over a 2 hour period) in the past month, (3) endorsed high emotion dysregulation (defined as scoring >46 on the 16-item Difficulties in Emotion Regulation Scale; Bjureberg, et al., 2016), and (4) lived in the U.S. Participants were excluded if they (1) were enrolled in psychotherapy and were unwilling to discontinue treatment, (2) have been diagnosed with Bipolar I or a psychotic disorder, (3) could not understand verbal or written English, or (4) did not have access to a computer with Internet.

Recruitment and Randomization

We recruited participants through online forums and classifieds (e.g. Reddit.com and Craigslist.org). Over the 150 recruiting days, 398 individuals contacted us via email or phone. Of those, 142 were assessed for eligibility, 90 were determined to be eligible and 60 were randomized (see Figure 1). One randomized participant who did not complete the baseline or follow-up assessments could not be included in the analyses, leaving 59 ITT participants. If participants were eligible, they were provided informed consent prior to randomization using an Institutional Review Board (IRB) approved consent form.

To account for potential covariates that may spuriously affect analytic results, participants were matched on biological sex (male, female), degree of suicide ideation (< 7 or = >7 on SBQ-R), and degree of disordered drinking (<16 or = >16 on the Alcohol Use Disorders Identification Test, AUDIT; Saunders et al., 1993) using a minimization randomization. The randomization algorithm was implemented by entering matching criteria information into a computerized program specifically designed for minimization randomization (MinimPy). Participants were randomized using a 1:1 ratio designed to allocate equal numbers of participants to the experimental condition (iDBT-ST; n = 31) as to the control condition (Waitlist; n = 29).

Procedures

The University of Washington’s IRB approved all study procedures. Regardless of condition assignment, all participants completed assessments at pre-treatment and monthly for four months totaling five assessment waves. Participants received Amazon gift codes via e-mail after completing each monthly assessment ($15 for baseline, $20 for 1-month, $25 for 2-month, and $30 for 3- and 4-month totally $120).

Drop-out

Participants were considered to have dropped out from the treatment once they stopped attempting or completing online sessions for at least 3 weeks in a row (e.g., did not log into to articulate.com for three weeks). Participants who did not complete sessions, but who completed subsequent sessions (e.g., missed sessions 2 and 3 but completed sessions 4–8) were not considered drop-outs. If someone indicated that s/he wanted to stop receiving sessions, we assessed whether they wanted to continue receiving paid monthly assessments. We considered individuals who either wanted to stop receiving sessions or stopped completing sessions, but completed paid monthly assessments as treatment drops, and those who stopped completing sessions and assessments as study drops.

Suicide Risk Protocol

Throughout the 16-week trial period, weekly questionnaires assessing suicide risk were emailed to all participants. They included questions on: 1) frequency of suicidal urges in the past week, 2) intensity of suicidal urges in the past week, 3) seriousness of acting out on suicidal urges in the past week, and 4) current suicidal urge, all using a 5-point Likert scale. Participants endorsing a rating of three or higher on any item, or a 2-point increase from the previous week, would be called and assessed for suicide risk. Additionally, the number for the National Suicide Prevention Lifeline was situated next to prompts on suicidality received by all participants.

Measures

Screening Measures

Demographic information

Demographic information was collected using the Demographic Data Schedule – Short Version (DDS; Linehan, 1982), which obtains a selection of demographic data including age, sex, ethnicity, marital status, income, educational level, and occupation. High concurrent validity was established comparing DDS responses to hospital chart data for a sample of psychiatric inpatients (Linehan, 1982).

Treatment history

Information related to participants’ current and past treatment use including hospitalizations, as well as medication use were obtained using the Treatment History Interview (Linehan & Heard, 1987).

Heavy episodic drinking

Binge or heavy episodic drinking was assessed using a single item from the NIAAA recommended questions (NIAAA, 2003). The specific question assessed for the frequency at which participants consumed 4 (for women) or 5 (for men) standard alcoholic beverages in a 2-hour period in the past 30-days. The single item of binge drinking has been validated as a predictor of problematic alcohol use (Canagasby & Vinson, 2005; Williams & Vinson, 2001).

Emotion dysregulation

Emotion dysregulation was assessed using the 16-item version of the Difficulty in Emotion Regulation Scale (DERS-16; Bjureberg, et al., 2016). The DERS-16 has shown excellent internal consistency (α = 0.92 to 0.95), good test-retest reliability (r = 0.85), and good convergent and discriminant validity in clinical and community samples. In addition, the DERS-16 retains similar convergent and discriminant validity when compared to the original 36-item DERS (Gratz and Roemer, 2004).

Suicide ideation

The presence of suicide ideation was assessed using the Suicide Behavior Questionnaire-Revised (SBQ-R; Osman et al., 2001), a 4-item measure that assesses suicidal ideation and the likelihood of future suicidal behavior. Suicidal ideation is defined as having thoughts about killing oneself. This corresponds with a score of 2 (“brief passing thought”) or more on Item 1 of the SBQ-R (“Have you ever thought about or attempted to kill yourself?”), which has been found to differentiate suicidal ideators and attempters from nonsuicidal individuals in clinical and non-clinical samples with maximal sensitivity (0.80 to 1) and specificity (0.96 to 1; Osman et al., 2001).

Acceptability, Feasibility, and Usability

Intervention Use

Treatment engagement was examined by analyzing the amount of time spent on each session, as well as adherence with treatment modes such as homework completion.

Perceived usefulness

Participants were asked about the usefulness of each session on a 7-point Likert scale at the end of each session. Higher scores indicate higher perceived usefulness.

Primary Outcomes

Suicidal ideation

During the outcome assessment portion of the study, suicidal ideation total score was assessed using the Scale for Suicidal Ideation (SSI; Beck et al., 1979). The SSI is a 19-item self-report assessment that addresses domains related to suicidal ideation. Each item is scored from 0–2, with higher total scores indicating higher suicide ideation. Individuals answering a 0 (“not at all”) on questions 4 (“desire to make an active suicide attempt is none”) and 5 (“would take precautions to save my life”) do not answer the subsequent 14 questions. The SSI has demonstrated moderate internal consistency (α = 0.84–0.89) and high interrater reliability (r = 0.83–0.98) (Beck et al., 1997; Beck et al., 1979).

Alcohol use

Two primary methods were used to assess alcohol use throughout the study. First, we used the Alcohol Use Disorders Identification Test total score (AUDIT; Saunders et al.1993), which is a 10-item self-administered measure specifically designed to identify hazardous and harmful alcohol consumption. The period was set for drinking behavior in the last 30-days. The AUDIT cutoff scores yield sensitivity scores that range from 0.90 to 0.80 and an average specificity score of 0.80 (Allen et al., 1997). Reinert and Allen (2007) calculated a median reliability of the AUDIT of 0.83 across 18 studies.

Alcohol quantity and frequency were also assessed using a self-administered web-based version of the Timeline Followback (TLFB; Rueger, et al., 2012). The TLFB has been shown to be a psychometrically valid instrument for obtaining retrospective daily estimates of alcohol consumption (Sobell & Sobell, 1992). Research has demonstrated that the self-administered web-based version is strongly correlated (r = 0.83 to 0.93) to the interview version of the TLFB. The TLFB was used as a measure for the total number of standard drinks each participant consumed during a thirty-day period (total drinks). The presence of HED at follow-up assessment was determined by dividing the number of standard drinks consumed by the number of hours spent drinking. Women drinking an average of 4 standard drinks and men drinking an average of 5 standard drinks in a two-hour period were considered to have engaged in a binge drinking episode.

Emotion dysregulation

Emotion dysregulation was assessed using the Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004). The DERS is a 36-item self-report measure that assesses individuals’ typical levels of emotion dysregulation. The DERS has been found to have high internal consistency (α = 0.93), good test-retest reliability (r = 0.88), and adequate construct and predictive validity. Previous research also supports the reliability and validity of the DERS in alcohol and substance users (Fox, Hong, & Sinha, 2008; Gratz & Tull, 2010). We used the 36-item total score for as the outcome variable in the analyses.

Treatment

iDBT-ST

The intervention was largely modeled from the in-person format of DBT skills training (e.g. Linehan, 2014) and further iterated from an earlier version computerized DBT for emotion regulation (iDBT-ER; Lungu, 2015). The intervention component of the study included 8 modules that were released to participants in active treatment over a course of 8 weeks. The first two weeks were focused on mindfulness skills (“What” and “How” skills), the following two weeks were focused on reducing problematic drinking (“Dialectical Drinking,” “Clear Mind,” “Community Reinforcement,” and “Burning Bridges”), the subsequent three weeks were focused on increasing emotion regulation strategies (“Model of Emotions,” “Check the Facts,” “Opposite Action,” “Problem Solving,” “Building Mastery,” and “Cope Ahead”), and finally, the last week included skills on tolerating distress (“TIP” & “Distract”). Each session included 2–3 new DBT skills. Each new skill was first introduced via a 3–7 minute video segment. After the video, participants would be directed to a “Key Point” page in which they were presented with the salient aspects from the video. Finally, participants would engage in the skill through an interactive and guided practice. At the end of each session, participants would select a homework practice. Participants were then emailed fillable DBT worksheets based on their chosen homework assignment and were encouraged to engage in skills practice via daily emails and/or text messaging. Each session lasted approximately 30–50 minutes. See Wilks et al., (2017) for a more in-depth explanation of the session contents.

Waitlist

Individuals randomized to the waitlist control condition received weekly questionnaires assessing the frequency, intensity, and seriousness of their suicide ideation as well as the frequency and quantity of alcohol consumption. Those endorsing high levels of suicide ideation received a call from clinic staff trained in suicide risk assessment. After 8-weeks, those randomized to waitlist would be given access to the intervention.

Statistical Methods

Descriptive data were used to evaluate treatment feasibility, safety, and acceptability. This study was not powered to test hypotheses about potential between-condition differences on primary outcomes. Therefore, emphasis was placed on evaluating indices of clinical significance and between- and within-group effect sizes were used to evaluate the magnitude of treatment effects. For continuous outcomes, between group effect size was calculated using the following formula: Effect size = β * time/SDraw, where β is the estimated coefficient of the difference in slope for each condition, SDraw is the pooled standard deviation between conditions at pretreatment, and time is the number of time points included in the analysis. Within group effect size was calculated using the following formula: Effect size = M1 – M2 / SDpooled, where M1 is the baseline mean score, M2 is the follow-up mean score, and SDpooled is the pooled standard deviation between time points. The resulting effect size is interpreted using Cohen’s specifications (1988). Cohen’s d is best suited to continuous outcomes in longitudinal analyses; therefore, it is only reported for the continuous outcomes. Reliable and clinically significant improvement were calculated per the criteria suggested by Jacobson and Truax (1991) in order to better understand the effects of both treatment and waiting on each outcome variable. Reliable change index (RCI) was calculated for each individual and outcome using the following formula: RC = x1 – x2/Sdiff , where x1 is the score at baseline, x2 is the follow-up score at a chosen time point, and Sdiff is the standard error of difference based on baseline standard deviation and Cronbach’s α. Clinically significant change (CSC) was operationalized as reaching a level of functioning, reflected in outcome scores, that was (1) closer to the mean score of the non-clinical population than that of the clinical population, or (2) where non-clinical normative data were unavailable, greater than two standard deviations outside the clinical range. Where normative standards have not been established, clinical and/or non-clinical norms were aggregated from publications of studies that included data on the relevant measures. Participants showing both reliable change (RCI>1.96) and clinically significant improvement were classified as “recovered”; those showing reliable change only were classified as “improved”; those showing neither were classified as “unchanged/deteriorated.” Mann-Whitney U analyses were conducted to evaluate between-group differences in the distribution of change classification at each monthly assessment point. The RC indices, CSC cut-offs, sources of reliability and norming data were as follows: SSI (RC = ±8.51, CSC ≥ 2.31; Beck et al. 1997; Chioqueta & Stiles, 2006; Pollock & Williams, 2004; Stepakoff, 1998; Sokero et al., 2003; Zhang & Brown, 2007); AUDIT (RC = ± 9.59, CSC ≤ 8.00; Hays et al., 1995; Saunders et al.,1993); DERS (RC = ± 19.40, CSC ≤ 81.84; Gratz & Chapman, 2007; Gratz & Roemer, 2004; Gratz & Roemer, 2008; Harrison et al., 2009; Tull et al., 2007).

All randomized participants who completed at least one monthly assessment were included in the longitudinal analyses. All observations were included at any time point giving a potential of 295 from 59 individuals. In these analyses, 259 observations were retained across all waves, or 87.80% of all possible observations. Continuous outcome variables (AUDIT, DERS, SSI) were analytically evaluated using multilevel modeling (e.g. HLM; Bryk & Raudenbush, 1992) with restricted maximum likelihood estimator (REML), which does not bias estimates with missing data. For outcomes with a preponderance of zeros (TLFB), Generalized Estimating Equations (GEE, Liang & Zeger, 1986; Zeger & Liang, 1986) with a negative binomial (NB) distribution with a log function was used. Using the NB approach, the resulting coefficients are exponentiated and interpreted as risk-ratios (RR), where values above or below one are interpreted as a percentage increase or decrease in the rate of the outcome. Both HLM and GEE can handle missing data at level 1, thus increasing statistical power. All analyses were conducted using SPSS version 19 (IBM Corp).

Results

Participant Characteristics

Of those who consented to our demographic survey (n = 56), participants were more likely to identify as female (69.5%) and Caucasian (82.1%). Income level among those who chose to disclose (n = 54) was generally low, with median reported past-year earnings falling at the minimum wage level of $15,000 to $19,999; the lowest range (less than $5,000) was also the most frequently endorsed (n = 12). Nearly half (53.3%) of ITT participants reported lifetime psychiatric hospitalization. Many of the participants (46.7%) endorsed receiving psychological treatment in the last year, while a majority (88.3%) had received psychotherapy sometime in their lifetime. Sample characteristics are displayed on Table 1.

Table 1.

Participant Demographics and Clinical Characteristics at Baseline by Condition (N=59)1

iDBT-ST (n = 30) Waitlist (n = 29) Total ITT (N = 59)
n (%) M SD n (%) M SD N (%) M SD Test Stat a p
Age 38.0 11.3 37.4 10.1 38.0 10.4 0.21 0.83
Sex 0.23 0.63
Male 10 (33.3) 8 (27.6) 18 (30.5)
Female 20 (66.7) 21 (72.4) 41 (69.5)
Racial background 1.93 0.62b
Caucasian 23 (79.3) 23 (85.2) 46 (82.1)
African American 4 (13.8) 4 (14.8) 8 (14.3)
Asian 2 (6.9) 0 2 (3.6)
Annual income 0.57 0.93b
< $10,000 8 (26.7) 8 (32.0) 16 (29.6)
$10,000–24,999 12 (41.4) 11 (44.0) 23 (42.6)
$25,000–50,000 7 (24.1) 4 (16.0) 11 (20.4)
≥$50,000 2 (6.9) 2 (8.0) 4 (7.4)
Mental Health Treatment
Past year (%yes) 48.4 44.8 46.6 0.08 0.78
Never (%) 12.9 10.3 11.6 0.95 1.00b
Psychiatric hospitalization
Lifetime (%yes) 51.6 55.2 53.4 0.08 0.78
Marital Status 11.17 0.01b, c
Married 1 (3.4) 8 (29.6) 9 (16.1)
Not married2 28 (96.9) 19 (70.4) 47 (83.9)
Children 3 (y) 10 (37.0) 13 (48.1) 23 (42.6) 0.68 0.41
Open in a new tab

Note.

1

Intend to Treat only. Of 56 participants who agreed to demographics, two declined to disclose income level.

2

Single, divorced, separated or widowed.

3

Biological children.

a

A t-test is reported to compare mean age and number of children between groups; for remaining comparisons, chi-square values are reported.

b

For chi-square comparisons with minimum expected cell count <5, the Fisher’s exact test is reported.

c

Difference in marital status, despite reaching significance, was not considered a confound as it did not explain significant variance on any of the outcome measures when added into the models.

Comparison of demographic variables between participants randomized to each condition revealed a statistically significant difference on marital state between conditions. Specifically, nearly 30% of individuals randomized to waitlist were married compared to 3.4% of those randomized to receive iDBT-ST immediately (p <.001). However, adding marital status (married versus not married) into the outcome models did not explain significant variance on any of the measures, and as such were not included in the outcome analyses presented below.

Treatment Retention and Feasibility

While many participants experienced little to no problems with the intervention platform, nearly half (45%) participants contacted study staff with a variety of technical and/or access issues. Most commonly, participants experienced technical problems with the program platform (n = 16), accessibility barriers (n = 13), and/or connectivity issues (i.e. slow/inconsistent Internet) (n = 4). The technical problems included inability to begin or move past session contents and freezing. Accessibility barriers were issues that interfered with participants’ ability to access and use the computerized program, and included being jailed (n = 2), computer/phone theft (n = 4), homelessness (n = 3), and/or moving (n = 2). The technical and access problems appear to pose a considerable barrier to treatment completion. Among individuals who dropped from treatment and provided a reason (n = 21), 52% cited technical problems, 33% cited access problems, and 9% cited connectivity problems. Regarding adherence to treatment procedures, 89.83% of individuals attempted session 1 and 84.47% of participants completed session 1. Session 8 was the least attempted and completed with only 30.51% of the ITT sample both attempting and completing the session.

Over the 8-week course of treatment, the number of individuals who went on to attempt the next session declined by an average of approximately 5 individuals per subsequent session (SD = 1.86). In addition, 61.65% of session attempters reported completing the assigned homework (SD = 6.95), and 94.30% of those who completed the homework found it to be effective (SD = 8.96).

Treatment Acceptability

During active treatment, participants were asked to rate their perceived usefulness of each session on a 0–5 point scale. Collapsed over the entire 8 sessions, the average usefulness rating was 3.91 (SD = 0.17). The participants rated session 4 as the most useful (M = 4.09, SD=0.90) followed by session 3 (M = 4.08, SD=1.23); these comprised the module targeting addiction. Sessions 1 and 2, which both target mindfulness, were rated as least useful (M = 3.69, SD = 1.18; M = 3.69, SD = 1.14). In addition, participants were assessed on the urge to quit over each session. Collapsed over all 8 sessions, the mean urge to quit was 1.79 (SD = 1.05), indicating that on average, users endorsed a low to moderate urge to quit.

Treatment Safety

As mentioned above, participants were assessed weekly on the frequency, intensity, and seriousness of suicidal urges. Individuals responding in a manner that would indicate high risk for suicide were contacted by study staff for further assessment. Collapsed across all time points, the participants endorsed experiencing suicidal thoughts less than half the days of the week (M = 1.74, SD = 0.82). On average, participants endorsed relatively low urge to die by suicide (Range 1–6, M = 2.17, SD = 1.23).

Throughout the entire study, weekly assessment responses prompted a total of 75 suicide risk assessment calls, nested among 35 individuals (Range = 1–8 calls, M = 2.14 calls per individual). While 35 individuals responded in such a way that prompted follow-up from study staff, only 25 successful contacts were made, nested among 16 participants. 50 “caring emails” were sent to those who could not be contacted by phone (see Wilks et al., 2017 for “caring email” script). To evaluate the safety of using a waitlist control with a suicidal sample, GEE analyses with a negative binomial distribution were conducted to investigate differences in severity of suicidality over time both within and between conditions. Compared to individuals on the waitlist, those randomized to immediate treatment saw significantly faster reductions in intensity of suicidal urges (RR = 0.95, p < .05).

Comparing iDBT-ST to Waitlist from baseline to month 2

Table 2 presents means and standard deviations for all outcome variables across all time points, Table 3 presents the HLM and GEE analyses for all outcome variables, Table 4 presents both within and between group effect sizes for all variables, and Table 5 presents results from the clinical significance analyses. From baseline to month 2, participants randomized to receive treatment immediately consumed significantly less alcohol than those who were on the waitlist during this period (RR = 0.81, p = .02). In addition, 67.7% of individuals reported having engaged in less than 2 episodes of HED in the previous month, which falls under the criteria for initial study eligibility, compared to 55.2% of individuals on the waitlist. While there were no significant between condition differences in SSI, AUDIT, or DERS, analyses of scores from baseline to month 2 showed a large between-group effect size for both SSI (d = −0.72) and AUDIT (d = −0.73), along with a small effect size for DERS (d = 0.34).

Table 2.

Means and standard deviations of outcome variables between condition

iDBT Waitlist
n M SD N M SD
SSI Baseline 30 13.54 7.52 29 14.53 7.95
1-month 24 9.59 7.09 26 13.14 7.93
2-month 22 8.67 6.80 28 12.48 8.37
3-month 24 7.38 7.34 26 11.23 9.14
4-month 24 5.45 6.62 26 9.59 8.99
AUDIT Baseline 30 22.83 9.24 29 22.03 7.54
1-month 24 17.38 7.78 26 21.21 10.17
2-month 22 16.64 7.32 28 19.71 10.18
3-month 24 16.59 9.31 26 16.15 9.41
4-month 24 13.62 8.86 25 16.46 8.77
TLFB (total drinks) Baseline 30 132.20 100.44 29 127.82 78.11
1-month 24 83.19 65.76 26 132.65 92.53
2-month 22 74.24 76.71 28 117.22 98.66
3-month 24 65.96 77.43 26 106.44 98.42
4-month 24 61.84 60.80 25 82.60 90.24
DERS(Total) Baseline 30 123.93 26.40 29 128.44 26.82
1-month 24 117.13 22.33 26 133.45 28.14
2-month 22 113.27 25.34 28 126.72 34.07
3-month 24 113.78 26.95 26 124.99 26.76
4-month 22 109.18 26.11 26 114.35 29.37
Open in a new tab

Table 3.

Between condition examination of outcomes from time 0 to time 2

Measure Parameter Stat SE 95% CI p
SSI Time −1.20 0.58 −2.36- −0.04 .04
Condition −1.04 1.96 −4.95–2.86 .60
Time*Condition −1.08 0.87 −2.80–0.64 .22
AUDIT Time −1.15 0.66 −2.45–0.17 .08
Condition 0.05 2.22 −4.30–4.39 .98
Time*Condition −2.05 1.12 −4.43–0.33 .09
TLFB (Total Drinks as risk ratios) Time 0.95 -- 0.85–1.07 .42
Condition 1.03 -- 0.13–1.45 .83
Time*Condition 0.81 -- 0.67–0.97 .02
DERS Time −0.83 1.99 −4.73–3.07 .67
Condition −6.71 6.90 −20.20–6.77 .33
Time*Condition −4.57 3.38 −11.20–2.05 .18
Open in a new tab

Table 4.

Within and Between-Group Effect Sizes for continuous outcomes

Within-Group Effect Sizes Between-Group Effect Sizes
iDBT-ST Waitlist iDBT-ST vs Waitlist
Outcome TX FU RBL TX T2 T4
SSI 0.81 0.62 0.25 0.32 −0.72 −1.06
AUDIT 0.75 0.37 0.26 0.34 −0.73 −0.22
DERS 0.42 0.15 0.06 0.39 0.34 0.20
Open in a new tab

Note. All effect sizes are reported Cohen’s D; TX=Treatment phase, FU=Follow-up,

RBL=Repeated Baseline, T2=Month 2 and end of treatment portion for iDBT-ST and beginning for treatment for waitlist, T4=Month 4 and end of follow-up period for iDBT-ST and end of treatment for waitlist.

Table 5.

Clinically significant change for primary and secondary outcomes by condition.

iDBT-ST
Waitlist
Ua p
Valid n No change/deteriorated N (%) Improved N (%) Recovered N (%) Valid n No change/deteriorated N (%) Improved N (%) Recovered N (%)
SSI 1-month 24 20 (83.3%) 3 (12.5) 1 (4.2) 26 24 (92.3) 0 2 (7.7) 287.0 0.29
2-month 22 17 (77.3) 2 (9.1) 3 (13.6) 28 23 (82.1) 3 (10.7) 2 (7.1) 290.5 0.62
3-month 24 14 (58.3) 4 (16.7) 6 (25.0) 26 18 (69.2) 4 (15.4) 4 (15.4) 274.0 0.39
4-month 24 15 (62.5) 2 (8.3) 7 (29.2) 26 19 (73.1) 2 (7.7) 5 (19.2) 277.0 0.41
AUDIT 1-month 24 20 (83.3) 3 (12.5) 1 (4.2) 26 24 (92.3) 2 (7.7) 0 283.0 0.32
2-month 22 15 (68.2) 5 (22.7) 2 (9.1) 28 25 (89.3) 2 (7.1) 1 (3.6) 243.5 0.07
3-month 24 16 (66.7) 5 (20.8) 3 (12.5) 26 20 (76.9) 2 (7.7) 4 (15.4) 287.0 0.54
4-month 24 12 (50.0) 6 (25.0) 6 (25.0) 26 19 (73.1) 5 (19.2) 2 (7.7) 231.0 0.07
DERS 1-month 24 21 (87.5) 3 (12.5) 0 26 26 (100) 0 0 273.0 0.07
2-month 22 16 (72.7) 5 (22.7) 1 (4.5) 28 24 (85.7) 3 (10.7) 1 (3.6) 269.0 0.27
3-month 24 17 (70.8) 6 (25.0) 1 (4.2) 26 23 (88.5) 3 (11.5) 0 255.5 0.11
4-month 24 18 (75.0) 5 (20.8) 1 (4.2) 26 19 (73.1) 3 (11.5) 4 (15.4) 297.5 0.71
Open in a new tab

Note:

a

Mann-Whitney U analyses were carried out to compare between conditions the proportion of participants who remained unchanged, and who showed improvement or recovery at 1-month, 2-months, 3-months, and 4-months.

Regarding clinically significant change, recovery rates obtained at month 2 for SSI (13.6% iDBT-ST, 7.7% waitlist), AUDIT (9.1% iDBT-ST, 3.6% waitlist), and DERS (4.5% iDBT-ST, 3.6% waitlist) reflect a similar pattern of greater change, as well as between-group differences on the SSI and AUDIT compared to DERS scores. Mann-Whitney U analyses revealed no significant between-group differences in the distribution of change classification on the SSI, AUDIT, and DERS measures.

Within-group rates of change across all time points for iDBT-ST

While the iDBT-ST group was enrolled in active treatment (time 0-time 2), they saw a significant decrease in SSI (B = −2.31, SE = 0.66, p = .001), AUDIT (B = −3.18, SE = 0.96, p = .003), TLFB (total drinks) (RR = 0.77, p < .001), and DERS (B = −5.63, SE = 2.18, p = .01). During the group’s follow-up period (time 2-time 4), participants continued to see slower but still significant reductions in SSI (B = −1.82, SE = 0.63, p = .006) and AUDIT (B = −1.73, SE = 0.84, p = .04). There were no significant reductions in TLFB (total drinks) (RR = 0.89, p = .11) or DERS (B = −2.48, SE = 3.05, p = .42) during the follow-up period. At the 4-month assessment wave, only 26.7% of individuals in the group that received treatment immediately still engaged in two episodes of HED in the previous month.

Individuals randomized to receive treatment immediately saw large effect size changes during the treatment period for SSI (d = 0.81) and AUDIT (d = 0.75), but only medium for DERS (d = 0.42). During the follow-up period, their scores reflected that the intervention had a medium effect on SSI (d = 0.62), but a small effect size for both AUDIT (d = 0.26) and DERS (d = 0.15).

Within-group rates of change across all time points for waitlist

For those randomized to the waitlist, the control group saw a significant decrease in SSI scores (B = −1.20, SE = 0.57, p = .04), but did not see significant reductions in AUDIT (B = −1.15, SE = 0.67, p = .10), TLFB (total drinks) (RR = 0.95, p = .42), or DERS (B = −1.15, SE = 1.71, p = .50) scores during their waiting period (time 0-time 2). When those on the waitlist were enrolled in active treatment (time 2-time 4), participants saw a significant reduction in SSI (B = −1.62, SE = 0.54, p = .004), AUDIT (B = −1.90, SE = 0.71, p = .01), TLFB (total drinks) (RR = 0.84, p = .04), and DERS (B = −5.68, SE = 2.41, p = .02) scores; however, at a greater magnitude than those who were randomized to receive treatment immediately. At the end of the 4-month assessment, 69.0% of waitlisted individuals engaged in less than two episodes of HED in the previous month, placing them below the initial study eligibility. Individuals randomized to the waitlist observed small effect sizes during the waiting period for SSI (d = 0.25), AUDIT (d = 0.24), and DERS (d = 0.06). During their respective treatment portion, individuals continued to see small effects for SSI (d = 0.32), AUDIT (d = 0.34) and DERS (d = 0.39).

Discussion

The present study examined the feasibility, acceptability, and preliminary efficacy of an Internet-delivered DBT skills training intervention for suicidal individuals who engage in HED and experience difficulties in emotion regulation. Importantly, this study appeared to be safe and the treatment moderately usable. While there was a significant treatment dropout rate, those who continued to use the intervention found it useful. Regarding efficacy, the sample had significant reductions in suicidal ideation, alcohol use severity, alcohol quantity, and emotion dysregulation over the course of the sixteen-week study, with faster reductions when participants were enrolled in active treatment. In addition, individuals randomized to receive treatment immediately saw significantly steeper reductions in alcohol quantity. Moreover, at the 4-month assessment wave, the proportion of individuals no longer meeting the initial study criterion for HED was larger in the group assigned to receive treatment immediately, as compared to waitlisted participants. Nevertheless, in this final survey, almost 70% of all participants reported not having engaged in past-month binge drinking, indicating that the majority of participants given immediate or delayed access to treatment no longer met the criterion for HED that had initially demarcated their eligibility for the study. Furthermore, participants enrolled in active treatment reported significantly less suicide ideation on their weekly surveys than those on the waitlist. There were no between condition differences in alcohol use severity, suicide ideation as reported on the SSI, or emotion dysregulation; nonetheless, participants randomized to receive treatment immediately had larger effect sizes during their respective treatment periods and there were large between condition effects for both AUDIT and suicide ideation. Finally, results from the clinical significance analyses indicated much need for improvement in the treatment for this population.

Unfortunately, many participants dropped due to technological reasons, and only one third of participants completed all eight sessions. Most participants dropped due to technological or accessibility problems rather than inapplicability, disinterest, and/or perceived uselessness. Of note, treatment dropout tends to be relatively high for online interventions (35%; Melville et al. 2010), and particularly high for addiction treatment in general (Brorson, et al., 2013), so the dropout rate in this study is not an aberration, but predictable. To reduce the likelihood of dropping, we suggest that Internet-delivered interventions be incredibly simple to use, brief, and more portable to increase treatment effects.

Alcohol use and suicide ideation were reduced by a larger magnitude than emotion dysregulation. Moreover, rates of clinically meaningful recovery established for alcohol use and suicide ideation exceeded that of emotion dysregulation for all except one assessment point (month 4 in waitlist group). One possible explanation for the limited movement in DERS is the lack of sufficient dose in emotion regulation skills. Only three sessions were allotted for emotion regulation skills, while individuals attending face-to-face skills training receive up to seven weeks of emotion regulation skills (Linehan, 2014). In addition, only 64.4% of the ITT sample accessed any of the sessions dedicated to emotion regulation skills. Taken together, these findings suggest that participants in this study may not have received an adequate dose of emotion regulation skills.

Limitations

There are several limitations that warrant discussion. Principally, while there was only one significant difference in the rate of change between conditions, this study was not powered to test for significant between group differences. To that point, the sample was heterogeneous with wide confidence intervals obtained across all outcomes, highlighting the need for replication with a larger sample. Regarding suicidality for instance, the criterion for inclusion was set at a low minimum; one just needed to have the thought enter their mind at least once in the past thirty days, and no maximum limit was imposed. In the struggle between internal and external validity, we landed on the latter, as we did not want to exclude someone for being too suicidal or not suicidal enough. This resulted in large sample standard deviations that raised the minimum threshold for statistical significance, potentially eroding meaningful indications of change. Nonetheless, as this is the only treatment outcome study for suicidal adults who engage in co-occurring HED, we wanted to cast a wide net to capture a representative sample of this population.

Another limitation is the use of a waitlist control for a comparison condition. One problem with a waitlist control design is that there is no control over what other activities participants do, such as enrolling in other treatments. Another problem is that participants who enter a treatment study that include a waitlist may be inherently biased due to the potential barriers they face toward receiving adequate treatment (Patterson et al. 2016). Due to the barriers that our participants may face for obtaining services, they may have been more likely to participate in our study than the general population. For mild psychological disorders, those on the waitlist tend to improve without treatment, making waitlist conditions problematic (Ahola, et al., 2016; Furukawa et al., 2014). Indeed, even with screening results indicating substantial psychological distress, those on the waitlist saw significant improvements while waiting for the intervention. In addition, in this trial participants in the waitlist were closely monitored and repeatedly contacted throughout the waiting period, however their enrollment in other services was not assessed during this time.

Another limitation was the level of treatment dropout, which underlines a setback in overall acceptability and feasibility of iDBT-ST for suicidal and heavy episodic drinkers. While only 11.67% of participants dropped from the study approximately two thirds of participants dropped out of treatment or did not complete multiple sessions throughout the active treatment period. Although the majority of participants in this study did log on to access the intervention at least once, many experienced technology-related problems. Ultimately, the combination of technological issues, lack of willingness to access the program, and behavioral instability was a combination of barriers that made treatment adherence too difficult for the participants to overcome. iDBT-ST needed to be incredibly acceptable or effective to overcome the barriers that the participants experienced.

A considerable limitation to this study is the intervention platform itself, in that it may align too closely with face-to-face skills training, thus diminishing the advantages of a technological mediated intervention. Schueller and colleagues (2013) have argued that web-based psychotherapy is a skeuomorphism, or derivative of face-to-face psychotherapy. Designing web-based therapies (and iDBT-ST) in a way that closely mirrors its face-to-face counterpart stymies much of the advantages of technology, and precludes a level of effectiveness comparable to in-person treatment. Technology has the capacity to be in the backpack, pocket, or even wrist of a client at all times, thus wielding substantial potential to promote nascent behaviors at key times. However, in keeping with the literature of computerized interventions, iDBT-ST was topographically identical to in-person skills training (i.e. mindfulness then homework review, followed by new skill teaching, practice, and homework assignment), without evidence on the salience of any aspect or component in building new behavior into participant behavioral repertoires. Consequently, iDBT-ST was both a watered-down imitation of in-person skills training and a technology-mediated behavior change tool.

Future Directions

The most important extension of this work is a continued focus on high-risk and underserved individuals. Technological collaborations (e.g. human-computer interaction, machine learning), which could ultimately increase access to treatment, augment face-to-face DBT, and increase knowledge about how DBT works should be fully exploited. Along those lines, a reasonable first step is to redesign iDBT-ST to be more portable (e.g. mobile application: mDBT). The finding of significant reductions in all outcomes from an Internet-delivered intervention underscores the promise awaiting continued treatment development in this area. Finally, any future work relies on collaboration from individuals across disciplines, such as computer science and human computer interaction to expand access and increase knowledge in both fields.

Highlights.

  • Internet delivered DBT skills training (iDBT-ST) for suicidal drinkers was evaluated

  • iDBT-ST was rated as useful and acceptable

  • iDBT-ST was safe to administer

  • When using iDBT-ST, individuals consumed less alcohol and were less suicidal

Acknowledgments

Funding

This work was supported by the National Institutes on Alcohol Abuse and Alcoholism [F31 AA 24658-01]; and the Alcohol and Drug Abuse Institute [ADAI-0315-2]. Registration: Clinical.gov NCT02932241, https://clinicaltrials.gov/ct2/show/NCT02932241; Date of registration: 10-Oct-2016

This research is supported by a National Research Service Award from the National Institute on Alcohol Abuse and Alcoholism (F31 AA 24658-01) as well as a grant from the Alcohol and Drug Abuse Institute (ADAI-0315-2). We would like to thank the individuals who participated in the study.

Role of funding source

National Research Service Award from the National Institute on Alcohol Abuse and Alcoholism (F31 AA 24658-01) as well as a grant from the Alcohol and Drug Abuse Institute (ADAI-0315-2) funded this study.

Footnotes

Conflict of interest

We wish to draw the attention of the Editor to the following facts which may be considered as potential conflicts of interest and to significant financial contributions to this work. The first author has been paid by Behavioral Tech to conduct training on Dialectical Behavior Therapy (DBT). The last author is the developer of DBT and receives royalties from Guilford press for books sold on DBT. In addition, the last author is the owner of Behavioral Tech, which conducts training on DBT. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us. We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property. We further confirm that any aspect of the work covered in this manuscript that has involved either experimental animals or human patients has been conducted with the ethical approval of all relevant bodies and that such approvals are acknowledged within the manuscript. We understand that the Corresponding Author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). She is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible by the Corresponding Author and which has been configured to accept email from wilksc@uw.edu.

Sincerely,

Chelsey R. Wilks, Anita Lungu, Sin Yee Ang, Brandon Matsumiya, Qingqin Yin, & Marsha M. Linehan

Contributors

CW is the principal investigator on the study and contributed to the design of the study, the study protocol, and preparation of this manuscript. AL contributed to the research design. SA is a research assistant on the study and contributed to data collection and preparation of this manuscript. BM is a research assistant on the study and contributed to data collection. QY is a research assistant on the study and contributed to data collection and preparation of this manuscript. ML is the faculty sponsor for the NIAAA grant that provided funding for this study and contributed to the design of the study.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Allen JP, Litten RZ, Fertig JB, Babor T. A review of research on the Alcohol Use Disorders Identification Test (AUDIT) Alcoholism: clinical and experimental research. 1997;21(4):613–619. doi: 10.1111/j.1530-0277.1997.tb03811.x. [DOI] [PubMed] [Google Scholar]
  2. Beck AT, Brown GK, Steer RA. Psychometric characteristics of the Scale for Suicide Ideation with psychiatric outpatients. Behaviour research and therapy. 1997;35(11):1039–1046. doi: 10.1016/S0005-7967(97)00073-9. [DOI] [PubMed] [Google Scholar]
  3. Beck AT, Kovacs M, Weissman A. Assessment of suicidal intention: the Scale for Suicide Ideation. Journal of consulting and clinical psychology. 1979;47(2):343. doi: 10.1037/0022-006X.47.2.343. [DOI] [PubMed] [Google Scholar]
  4. Bjureberg J, Ljótsson B, Tull MT, Hedman E, Sahlin H, Lundh LG, … Gratz KL. Development and Validation of a Brief Version of the Difficulties in Emotion Regulation Scale: The DERS-16. Journal of psychopathology and behavioral assessment. 2016;38(2):284–296. doi: 10.1007/s10862-015-9514-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Brady J. The association between alcohol misuse and suicidal behaviour. Alcohol and Alcoholism. 2006;41(5):473–478. doi: 10.1093/alcalc/agl060. [DOI] [PubMed] [Google Scholar]
  6. Brown MZ, Comtois KA, Linehan MM. Reasons for suicide attempts and nonsuicidal self-injury in women with borderline personality disorder. Journal of abnormal psychology. 2002;111(1):198. doi: 10.1037/0021-843X.111.1.198. [DOI] [PubMed] [Google Scholar]
  7. Bruffaerts R, Demyttenaere K, Hwang I, Chiu WT, Sampson N, Kessler RC, … Florescu S. Treatment of suicidal people around the world. The British Journal of Psychiatry, bjp-bp. 2011 doi: 10.1192/bjp.bp.110.084129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Canagasaby A, Vinson DC. Screening for hazardous or harmful drinking using one or two quantity–frequency questions. Alcohol and Alcoholism. 2005;40(3):208–213. doi: 10.1093/alcalc/agh156. [DOI] [PubMed] [Google Scholar]
  9. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. Web-based Injury Statistics Query and Reporting System (WISQARS) 2016 Retrieved February 10, 2017 from www.cdc.gov/injury/wisqars/index.html.
  10. Conner KR, Pinquart M, Gamble SA. Meta-analysis of depression and substance use among individuals with alcohol use disorders. Journal of substance abuse treatment. 2009;37(2):127–137. doi: 10.1016/j.jsat.2008.11.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Fox HC, Hong KA, Sinha R. Difficulties in emotion regulation and impulse control in recently abstinent alcoholics compared with social drinkers. Addictive behaviors. 2008;33(2):388–394. doi: 10.1016/j.addbeh.2007.10.002. [DOI] [PubMed] [Google Scholar]
  12. Gratz KL, Roemer L. Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the difficulties in emotion regulation scale. Journal of psychopathology and behavioral assessment. 2004;26(1):41–54. doi: 10.1023/B:JOBA.0000007455.08539.94. [DOI] [Google Scholar]
  13. Gratz KL, Tull MT. The relationship between emotion dysregulation and deliberate self-harm among inpatients with substance use disorders. Cognitive therapy and research. 2010;34(6):544–553. doi: 10.1007/s10608-009-9268-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Han B, Compton WM, Gfroerer J, McKeon R. Prevalence and correlates of past 12-month suicide attempt among adults with past-year suicidal ideation in the United States. The Journal of clinical psychiatry. 2015;76(3):295–302. doi: 10.4088/JCP.14m09287. [DOI] [PubMed] [Google Scholar]
  15. Harrison A, Sullivan S, Tchanturia K, Treasure J. Emotion recognition and regulation in anorexia nervosa. Clinical psychology & psychotherapy. 2009;16(4):348–356. doi: 10.1002/cpp.628. [DOI] [PubMed] [Google Scholar]
  16. Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, … Sutcliffe P. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health technology assessment. 2006;10(33):1–186. doi: 10.3310/hta10330. [DOI] [PubMed] [Google Scholar]
  17. Kelly TM, Lynch KG, Donovan JE, Clark DB. Alcohol use disorders and risk factor interactions for adolescent suicidal ideation and attempts. Suicide and Life-Threatening Behavior. 2001;31(2):181–193. doi: 10.1521/suli.31.2.181.21512. [DOI] [PubMed] [Google Scholar]
  18. Lai MH, Maniam T, Chan LF, Ravindran AV. Caught in the web: a review of web-based suicide prevention. Journal of medical Internet research. 2014;16(1) doi: 10.2196/jmir.2973. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Linehan M. Cognitive-behavioral treatment of borderline personality disorder. Guilford press; 1993. [Google Scholar]
  20. Linehan M. DBT skills training manual. Guilford Publications; 2014. [Google Scholar]
  21. Linehan MM. Unpublished manuscript. University of Washington; 1982. Demographic data schedule (DDS) [Google Scholar]
  22. Linehan MM, Heard HL. Treatment history interview (THI) University of Washington; Seattle: 1987. [Google Scholar]
  23. Linehan MM, Wilks CR. The course and evolution of dialectical behavior therapy. American journal of psychotherapy. 2015;69(2):97–110. doi: 10.1176/appi.psychotherapy.2015.69.2.97. [DOI] [PubMed] [Google Scholar]
  24. Linehan MM, Korslund KE, Harned MS, Gallop RJ, Lungu A, Neacsiu AD, … Murray-Gregory AM. Dialectical behavior therapy for high suicide risk in individuals with borderline personality disorder: a randomized clinical trial and component analysis. JAMA psychiatry. 2015;72(5):475–482. doi: 10.1001/jamapsychiatry.2014.3039. [DOI] [PubMed] [Google Scholar]
  25. Lungu A. Doctoral dissertation. 2015. Computerized Trans-Diagnostic Dialectical Behavior Therapy Skills Training for Emotion Dysregulation. [Google Scholar]
  26. McManama O'Brien KH, Becker SJ, Spirito A, Simon V, Prinstein MJ. Differentiating adolescent suicide attempters from ideators: Examining the interaction between depression severity and alcohol use. Suicide and life-threatening behavior. 2014;44(1):23–33. doi: 10.1111/sltb.12050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. National Institute on Alcohol Abuse and Alcoholism. Task force on recommended questions of the National Council on Alcohol Abuse and Alcoholism: Recommended sets of alcohol consumption questions. 2003 Retrieved February 10, 2017 from http://www.niaaa.nih.gov/about/TaskForceQuestions.htm.
  28. Nock MK, Hwang I, Sampson NA, Kessler RC. Mental disorders, comorbidity and suicidal behavior: results from the National Comorbidity Survey Replication. Molecular psychiatry. 2010;15(8):868. doi: 10.1038/mp.2009.29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Nordström P, Schalling D, Asberg M. Temperamental vulnerability in attempted suicide. Acta Psychiatrica Scandinavica. 1995;92(2):155–160. doi: 10.1111/j.1600-0447.1995.tb09560.x. [DOI] [PubMed] [Google Scholar]
  30. Osman A, Bagge CL, Gutierrez PM, Konick LC, Kopper BA, Barrios FX. The Suicidal Behaviors Questionnaire-Revised (SBQ-R): validation with clinical and nonclinical samples. Assessment. 2001;8(4):443–454. doi: 10.1177/107319110100800409. [DOI] [PubMed] [Google Scholar]
  31. Patel V, Chisholm D, Parikh R, Charlson FJ, Degenhardt L, Dua T, … Lund C. Addressing the burden of mental, neurological, and substance use disorders: key messages from Disease Control Priorities. The Lancet. 2016;387(10028):1672–1685. doi: 10.1016/S0140-6736(15)00390-6. [DOI] [PubMed] [Google Scholar]
  32. Patterson B, Boyle MH, Kivlenieks M, Van Ameringen M. The use of waitlists as control conditions in anxiety disorders research. Journal of psychiatric research. 2016;83:112–120. doi: 10.1016/j.jpsychires.2016.08.015. [DOI] [PubMed] [Google Scholar]
  33. Reinert DF, Allen JP. The alcohol use disorders identification test: an update of research findings. Alcoholism: Clinical and Experimental Research. 2007;31(2):185–199. doi: 10.1111/j.1530-0277.2006.00295.x. [DOI] [PubMed] [Google Scholar]
  34. Rooke S, Thorsteinsson E, Karpin A, Copeland J, Allsop D. Computer-delivered interventions for alcohol and tobacco use: a meta-analysis. Addiction. 2010;105(8):1381–1390. doi: 10.1111/j.1360-0443.2010.02975.x. [DOI] [PubMed] [Google Scholar]
  35. Rueger SY, Trela CJ, Palmeri M, King AC. Self-administered web-based timeline followback procedure for drinking and smoking behaviors in young adults. Journal of Studies on Alcohol and Drugs. 2012;73(5):829–833. doi: 10.15288/jsad.2012.73.829. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Saunders JB, Aasland OG, Babor TF, De la Fuente JR, Grant M. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption- II. Addiction. 1993;88(6):791–804. doi: 10.1111/j.1360-0443.1993.tb02093.x. [DOI] [PubMed] [Google Scholar]
  37. Schneidman ES. Suicide as psychache: A clinical approach to self-destructive behavior. London: Rowan & Littelefield publishers; 1993. [Google Scholar]
  38. Sher L. Alcohol consumption and suicide. Qjm. 2005;99(1):57–61. doi: 10.1093/qjmed/hci146. [DOI] [PubMed] [Google Scholar]
  39. Slee N, Garnefski N, Spinhoven P, Arensman E. The Influence of Cognitive Emotion Regulation Strategies and Depression Severity on Deliberate Self-Harm. Suicide and Life-Threatening Behavior. 2008;38(3):274–286. doi: 10.1521/suli.2008.38.3.274. [DOI] [PubMed] [Google Scholar]
  40. Slee N, Spinhoven P, Garnefski N, Arensman E. Emotion regulation as mediator of treatment outcome in therapy for deliberate self-harm. Clinical Psychology & Psychotherapy. 2008;15(4):205–216. doi: 10.1002/cpp.577. [DOI] [PubMed] [Google Scholar]
  41. Sobell LC, Sobell MB. Measuring alcohol consumption. Humana Press; 1992. Timeline follow-back; pp. 41–72. [Google Scholar]
  42. Sokero T, Melartin T, Rytsala H, Leskela U. Suicidal ideation and attempts among psychiatric patients with major depressive disorder. The Journal of Clinical Psychiatry. 2003;64(9):1094. doi: 10.4088/jcp.v64n0916. [DOI] [PubMed] [Google Scholar]
  43. Stepakoff S. Effects of sexual victimization on suicidal ideation and behavior in US college women. Suicide and Life-Threatening Behavior. 1998;28(1):107–126. doi: 10.1111/j.1943-278X.1998.tb00630.x. [DOI] [PubMed] [Google Scholar]
  44. Tull MT, Barrett HM, McMillan ES, Roemer L. A preliminary investigation of the relationship between emotion regulation difficulties and posttraumatic stress symptoms. Behavior Therapy. 2007;38(3):303–313. doi: 10.1016/j.beth.2006.10.001. [DOI] [PubMed] [Google Scholar]
  45. Valentine SE, Bankoff SM, Poulin RM, Reidler EB, Pantalone DW. The use of dialectical behavior therapy skills training as stand-alone treatment: A systematic review of the treatment outcome literature. Journal of clinical psychology. 2015;71(1):1–20. doi: 10.1002/jclp.22114. [DOI] [PubMed] [Google Scholar]
  46. White A, Kavanagh D, Stallman H, Klein B, Kay-Lambkin F, Proudfoot J, … Young R. Online alcohol interventions: a systematic review. Journal of medical Internet research. 2010;12(5) doi: 10.2196/jmir.1479. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Whiteside U. PhD dissertation. Seattle, WA: University of Washington; 2011. A brief motivational intervention incorporating DBT skills for depressed and anxious young drinkers. [Google Scholar]
  48. Wilks CR, Coyle TN, Krek M, Lungu A, Andriani K. Suicide Ideation and Acceptability Toward Online Help-Seeking. Suicide and life-threatening behavior. 2017 doi: 10.1111/sltb.12356. [DOI] [PubMed] [Google Scholar]
  49. Wilks CR, Zieve GG, Lessing HK. Are trials of computerized therapy generalizable? A multidimensional meta-analysis. Telemedicine and e-Health. 2016;22(5):450–457. doi: 10.1089/tmj.2015.0129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wilks CR, Yin Q, Ang YS, Matsumiya B, Lungu A, Linehan MM. Internet-delivered dialectical behavioral therapy skills training for suicidal and heavy episodic drinkers: Protocol and preliminary results for a randomized controlled trial. JMIR Research Protocols. 2017;6(10):e207. doi: 10.2196/resprot.7767. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Williams R, Vinson DC. Validation of a single screening question for problem drinking. Journal of Family Practice. 2001;50(4):307–307. [PubMed] [Google Scholar]
  52. Zhang J, Brown GK. Psychometric properties of the scale for suicide ideation in China. Archives of Suicide Research. 2007;11:203–210. doi: 10.1080/13811110600894652. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES