Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Apr 23.
Published in final edited form as: Behav Res Ther. 2014 Oct 23;63:139–146. doi: 10.1016/j.brat.2014.10.003

Only Connect: The Working Alliance in Computer-Based Cognitive Behavioral Therapy

Brian D Kiluk 1, Kelly Serafini 1, Tami Frankforter 1, Charla Nich 1, Kathleen M Carroll 1
PMCID: PMC4408209  NIHMSID: NIHMS637795  PMID: 25461789

Abstract

The limited role of therapists in some technology-based interventions raises questions as to whether clients may develop a ‘working alliance’ with the program, and the impact on relationships with a therapist and/or treatment outcomes. In this study, the Working Alliance Inventory (WAI), and an adapted version for technology-based interventions (WAI-Tech), were administered within a subsample (n = 66) of cocaine-dependent individuals participating in a randomized trial evaluating the efficacy of Computer-Based Training for Cognitive Behavioral Therapy (CBT4CBT) as an adjunct to treatment as usual (TAU). Results suggest the WAI-Tech has relatively similar psychometric characteristics as the standard WAI; however the ‘bond’ subscale scores were lower on the WAI-Tech [F(1,52) = 5.78, p<.05]. Scores on the WAI-Tech were not associated with cocaine use outcomes, whereas total scores on the WAI for those assigned to TAU were associated with the percentage of days abstinent from cocaine (r = .43, p < .05). There was little evidence that adding a technology-based intervention adversely affected the working alliance with a therapist in this sample. These preliminary findings suggest some concepts of working alliance may apply to computer-based CBT, yet the function of the alliance may be different in technology-based interventions than in face-to-face psychotherapies.

Keywords: Working Alliance, Therapeutic Alliance, Technology-Based Interventions, Substance Use Treatment, CBT4CBT

INTRODUCTION

Multiple challenges to dissemination of evidence-based therapies, such as cognitive behavioral therapy (CBT), have spurred development of numerous technology-based interventions (see reviews, Cuijpers et al., 2009; Kaltenthaler et al., 2006; Marks et al., 2009; Richardson, Stallard, & Velleman, 2010; Spek et al., 2007). Technology-based interventions offer many potential benefits, such as providing broader access to empirically supported treatments, consistency in treatment delivery (i.e., increased fidelity), decreasing the demands on clinician time and clinic resources, and cost effectiveness (Carroll & Rounsaville, 2010; Marks & Cavanagh, 2009; Marsch & Dallery, 2012). For the treatment of substance use disorders, many technology-based interventions have demonstrated positive treatment outcomes (e.g., Bickel, Marsch, Buchhalter, & Badger, 2008; Budney et al., 2011; Carroll et al., 2008; Gustafson et al., 2011; Hester, Delaney, & Campbell, 2011; Ondersma, Chase, Svikis, & Schuster, 2005). However, there has been relatively little research regarding how these technology-based interventions may lead to changes in substance use (i.e., mediators/mechanisms of action), the extent to which findings regarding active ingredients of the parent therapy pertain to the technology-based intervention, and what treatment factors are predictive of better outcomes.

In traditional clinician-delivered interventions, the working alliance (also referred to as the therapeutic alliance) is one of the most consistent predictors of positive treatment outcomes (Horvath, Del Re, Fluckiger, & Symonds, 2011; Martin, Garske, & Davis, 2000). In the treatment of substance use disorders, a positive working alliance early in treatment has been associated with greater engagement, retention, and early improvements in substance use (Gibbons et al., 2010; Ilgen, McKellar, Moos, & Finney, 2006; Meier, Barrowclough, & Donmall, 2005; Meier, Donmall, McElduff, Barrowclough, & Heller, 2006). However, the relationship between alliance and outcome is complex, as reports of the robustness of the alliance as a predictor of treatment outcomes has been somewhat mixed after accounting for prior symptom change (e.g., Barber, Connolly, Crits-Christoph, Gladis, & Siqueland, 2000; Falkenstrom, Granstrom, & Holmqvist, 2014; Strunk, Brotman, & DeRubeis, 2010; Webb et al., 2011). Furthermore, several studies have found that therapist variability in the alliance (i.e. variability between therapists) rather than patient variability (i.e., variability within therapists), more strongly relates to treatment outcomes (Baldwin, Wampold, & Imel, 2007; Crits-Christoph et al., 2009). Although there is some disagreement regarding the magnitude of therapist effects, multilevel models applied to clinical trial data have shown that 5–10% of the total variability in outcomes is attributable to between-therapists differences (Crits-Christoph et al., 1991; Elkin, Falconnier, Martinovich, & Mahoney, 2006; Kim, Wampold, & Bolt, 2006). Because many technology-based interventions significantly alter the role of the therapist (with treatment implemented either independently from the therapist or significantly reduced amount of therapist contact), it is unclear how this change in delivery might affect important treatment processes such as the working alliance.

As an emerging area of research, there are few studies that specifically address the concept of a working alliance in technology-based interventions. Of those that have examined the working alliance within this context, the predominant focus has been on the alliance with a clinician guiding or providing the technology-based intervention, rather than a working alliance with the technology-based program itself. Most studies indicate alliance ratings with a clinician/therapist guiding/providing the technology-based intervention are in line with ratings found in face-to-face therapies (Andersson et al., 2012; Cook & Doyle, 2002; Kay-Lambkin, Baker, Lewin, & Carr, 2011; Knaevelsrud & Maercker, 2007; Preschl, Maercker, & Wagner, 2011; Sucala et al., 2012). However, a pilot study explored participants’ alliance ratings with a computerized CBT package for depression using a modified version of the Agnew Relationship Measure (Agnew-Davies, Stiles, Hardy, Barkham, & Shapiro, 1998), with the word “package” replacing “therapist” in the original scale, and found participants’ average ratings of alliance with the package were above the neutral midpoint, suggesting a positive relationship (Ormrod, Kennedy, Scott, & Cavanagh, 2010). Also, although significant decreases in depression were found after receiving the computerized CBT package, these outcomes were not related to the alliance with the computerized package (Ormrod et al., 2010). These were some of the first known reported results regarding a potential alliance with a computerized CBT program, as well as its relation to treatment outcomes, yet the study was limited by a small sample (n=16) and did not include a comparison condition.

Overall, relatively little is known about whether the concept of the working alliance is relevant to technology-based interventions, how this potential alliance may affect the relationship with a therapist, and whether it influences outcome in a manner similar to that of traditional therapist-client working alliance. There have been reports that some clients do describe a form of relationship with computerized interventions (Bickmore, Gruber, & Picard, 2005; Bickmore, Caruso, Clough-Gorr, & Heeren, 2005; Kaplan, Farzanfar, & Friedman, 2003; Ormrod et al., 2010). For example, Bickmore, Gruber, & Picard (2005) reported a computerized intervention for physical activity adoption that included a ‘relational agent’ (e.g., animated computer character that simulated face-to-face conversation using social-emotional behaviors) produced higher working alliance ratings than a comparison intervention without the relational qualities. Furthermore, results of qualitative interviews from participants engaged in a telephone-based health behavior intervention indicated users described the system in ways indicative of having a personal relationship with it (e.g., “friend”, “helper”, “mentor”) (Kaplan, Farzanfar, & Friedman, 2003). While these findings suggest clients may develop some form of working alliance with technology-based interventions, the nature of the alliance and how it may differ from traditional features of a working alliance is relatively unexplored.

To investigate this concept, we adapted a widely-used and well validated measure of the working alliance (WAI; Horvath & Greenberg, 1989) and implemented it in the context of a randomized trial evaluating the effectiveness of Computer-Based Training for Cognitive Behavioral Therapy (CBT4CBT; Carroll, Kiluk, Nich, Gordon, et al., 2014). The purpose of the present study was to: (1) provide a preliminary psychometric evaluation of the newly adapted version of the WAI designed to measure the alliance with a technology-based intervention (WAI-Tech), including reliability and construct validity; (2) to evaluate the extent to which using a technology-based intervention as an adjunct to standard treatment might affect participants’ reported alliance with their clinicians; and (3) explore the contribution of a working alliance with a technology-based intervention to substance use treatment outcomes, such as treatment retention and frequency of substance use.

METHODS

Overview of the study: Treatments, participants, and assessment schedule

As described in detail in the main study report (Carroll, Kiluk, Nich, Gordon, et al., 2014), 101 cocaine-dependent individuals enrolled in an outpatient methadone program were randomized to one of two treatment conditions for a period of 8-weeks: (1) standard methadone maintenance (‘treatment as usual’, TAU) or, (2) TAU plus CBT4CBT. The TAU condition consisted of daily methadone maintenance along with weekly group and/or individual sessions with a substance use counselor. Those randomized to the CBT4CBT condition were also provided with weekly access to the computer program in a small private room within the clinic. Briefly, CBT4CBT (Carroll et al., 2008; Carroll et al., 2009) is a computer-based version of CBT for substance use disorders (Carroll, 1998) that uses videos, games, cartoons, and interactive exercises to teach CBT concepts and coping skills in an engaging manner. It includes 7 ‘modules’ that cover a specific CBT skill/topic area, with each module taking approximately 45 minutes to complete. It is highly user-friendly, requires no previous experience with computers and no reading skills, as all material presented in text is read aloud by a narrator. A research staff member guided participants through their initial use of the CBT4CBT program and was available to answer questions each time participants accessed the program. Participants were assessed before treatment, twice weekly during treatment, and at the 8-week treatment termination point, as well as at several time points following treatment termination (1-, 3-, and 6-months after termination). Post-treatment interviews were obtained from 98 of the 101 individuals randomized (97%); complete follow-up data were available for 93 of those randomized (92%).

Individuals were eligible who met criteria for current (past 30 days) cocaine dependence. Individuals were excluded only if (1) they failed to meet Diagnostic and Statistical Manual – Fourth Edition (DSM-IV; American Psychiatric Association, 1994) criteria for current cocaine dependence, (2) had an untreated/unstabilized psychotic disorder or had current suicidal/homicidal ideation such that more intensive treatment was needed, or (3) could not read at a 6th grade level in order to provide written informed consent and complete study assessments. All participants provided informed consent and the procedures followed were in accord with the standards of the Yale University School of Medicine Human Investigations Committee.

The Substance Use Calendar (Carroll et al., 2004) was used to assess substance use, which is a calendar-based assessment of self-reported substance use similar to the Timeline Follow Back (Fals-Stewart, O'Farrell, Freitas, McFarlin, & Rutigliano, 2000; Sobell & Sobell, 1992). Participant self-reports of drug use/abstinence were verified through urine toxicology screens that were obtained at every assessment visit. Rates of discordance between participant self-report and urine toxicology result (i.e., urine result positive for cocaine, yet participant denied cocaine use during the previous 3-day period) were fairly low (12% of the 875 urine specimens collected).

Results from the main study evaluating the efficacy of CBT4CBT indicated participants assigned to the CBT4CBT condition (N=47) demonstrated better cocaine use treatment outcomes than those assigned to TAU only (N=54) (Carroll, Kiluk, Nich, Gordon, et al., 2014). Specifically, those assigned to CBT4CBT were significantly more likely to achieve 3 or more consecutive weeks of abstinence from cocaine (36% compared to 17%; p<.05), and also demonstrated a greater reduction in the frequency of cocaine use over the course of treatment and through a 6-month follow-up period compared to those assigned to TAU only. These results replicated initial findings regarding the efficacy and durability of CBT4CBT as an add-on to standard treatment at reducing rates of drug use compared to standard treatment only (Carroll et al., 2008; Carroll, Kiluk, Nich, Gordon, et al., 2014; Carroll et al., 2009).

Development of the WAI-Tech

The Working Alliance Inventory (WAI; Horvath & Greenberg, 1989) was selected as the starting point for the development of an instrument evaluating the working alliance in technology-based interventions, as it (1) is a widely used measure with very strong psychometric properties (Horvath & Greenberg, 1986; Horvath & Symonds, 1991; Tichenor & Hill, 1989; Tracey & Kokotovic, 1989), (2) has been shown to predict outcome in a variety of populations, including substance users (Cecero, Fenton, Nich, Frankforter, & Carroll, 2001; Connors, Carroll, DiClemente, Longabaugh, & Donovan, 1997; Connors, DiClemente, Derman, Kadden, & Carroll, 2000; Fenton, Cecero, Nich, Frankforter, & Carroll, 2001), and (3) reflects the broad conceptualization of the alliance that is consistent with Bordin’s pantheoretical definition (Bordin, 1979).

The WAI is a 36-item paper-pencil instrument designed to measure the level of alliance between client and therapist along three dimensions: Task (12 items), Bond (12 items), and Goal (12 items). The task subscale indicates how responsive the therapist was to the client’s focus or need. The goals subscale refers to the extent to which goals were important, mutual, and capable of being accomplished. The bonds subscale refers to the degree of mutual liking and attachment as assessed through such means as tone of voice, empathy, and comfort in exploring intimate issues. The client version contains items including a blank space for participants to mentally insert the name of their therapist (e.g., “It seems as if _________ and I understand each other.” Participants rate their responses using a 7-point scale (1 = “never” to 7 = “always”). The WAI yields three 12-item subscale scores (Task, Bond, Goal) and one overall score (Total). All participants in this study completed the WAI (client version) reflecting their relationship with their substance use counselor at the treatment facility, following session 2, 4, and 8 during the 8-week course of treatment.

The Working Alliance Inventory for Technology-Based Interventions (WAI-Tech) was based closely on the WAI, including a parallel set of 36 items and using the 7-point rating scale with identical anchors. The majority of items were not changed from the original WAI; the major difference was that participants were instructed to insert the name of the computer program in the blanks: e.g., “I am confident in the CBT4CBT program’s ability to help me”. Several items, most often those from the bond scale, were reworded to preserve comprehension while retaining intent as much as possible. For example, “I believe _____ is genuinely concerned with my welfare” was changed to “I believe ____ is genuinely relevant to my wellbeing”. Instructions specified that items referred to the computer program only, not the computer itself, or the research team. Only those participants assigned to the CBT4CBT condition completed the WAI-Tech following session 2, 4, and 8 with the computer.

Evaluation of Treatment

Upon completion of the 8-week treatment period, all randomized participants completed a 10-item self-report questionnaire rating their satisfaction (5-point scale from “very dissatisfied” to “very satisfied”) with the type of treatment received, satisfaction with their counselor, how much they have changed during treatment (5-point scale from “I’m much worse” to “I’m much better”), and how much they attributed changes in functioning to the treatment received (5-point scale from “definitely not related” to “definitely related”).

Those assigned to CBT4CBT also completed an assessment of their satisfaction with and experience of the program using a scale developed previously (Carroll et al., 2008). This scale consisted of 24 items assessing the participants’ impression of various aspects of the CBT4CBT program. (e.g., I like the narrator for this program; I can relate to the characters in the videos). These were scored using a similar 7-point scale using the same anchors as the WAI.

Statistical Analyses

Participants assigned to the CBT4CBT condition who completed the WAI-Tech were compared to participants assigned to TAU who completed a WAI using Chi-square and Analysis of Variance (ANOVA) to evaluate differences according to demographic characteristics and baseline substance use severity. Cronbach’s alpha was used to evaluate the internal consistency of the WAI-Tech Total and subscale scores. These were compared to Cronbach’s alpha for the standard WAI. To evaluate construct validity, Pearson product moment correlations were calculated between WAI and WAI-Tech at each time point, as well as with participant ratings of satisfaction with the CBT4CBT program and with treatment. ANOVAs were used to compare mean scores for the WAI-Tech and WAI at each time point. Random effects regressions were used to explore the stability of WAI-Tech scores over time, and whether there was differential change compared to the WAI. Lastly, correlations were used to explore the relationship between the WAI-Tech and several common substance use treatment outcomes (Carroll, Kiluk, Nich, DeVito, et al., 2014; Donovan et al., 2012), such as the days retained in treatment, the percentage of days of self-reported abstinence from cocaine, the percentage of cocaine negative urine samples submitted, and whether the participant achieved at least 21- consecutive days of cocaine abstinence. Multiple regression and structural equation modeling (SEM) were used to further explore the relationship between WAI-Tech and treatment outcomes.

RESULTS

Participants

Of the 47 participants assigned to the CBT4CBT condition, 34 completed the WAI-Tech at least once (development and implementation of the WAI-Tech began after the trial initiated enrollment). Table 1 displays the demographic and baseline characteristics of these participants. There were no baseline differences between those assigned to CBT4CBT who completed the WAI-Tech and those assigned to TAU who completed a WAI (n = 32). The sample of participants who completed the WAI-Tech consisted of an equal number of males and females (50% each), mostly Caucasian (56%), never married (85%), and with an average age of 43.4 (SD = 9.6). The majority completed high school (74%), were unemployed (97%), and were receiving public assistance (77%). Most (71%) reported using a computer regularly. The average number of days of cocaine use during the month prior to treatment entry was 15.2 (SD = 9.2).

Table 1.

Demographic and baseline characteristics across groups

CBT4CBT
(n=34)		 TAU
(n=32)		 Total
(n=66)		 χ2*
Categorical variables n (%) n (%) n (%)
Female 17 (50) 16 (50) 33 (50) 0
Race / Ethnicity 3.33
   Caucasian 19 (55.9) 19 (59.9) 38 (57.6)
   African-American 13 (38.2) 8 (25) 21 (31.8)
   Hispanic 2 (5.9) 3 (9.4) 5 (7.6)
   Other 0 1 (3.1) 1 (1.5)
   Multiracial 0 1 (3.1) 1 (1.5)
Completed High School 25 (73.5) 23 (71.9) 48 (72.7) 0.02
Never married/living alone 29 (85.3) 29 (90.6) 58 (87.9) 0.44
Unemployed 33 (97.1) 28 (87.5) 61 (92.4) 2.15
Referred by criminal justice system 7 (20.6) 7 (21.9) 14 (21.2) 0.02
On Public Assistance 26 (76.5) 26 (81.3) 52 (78.8) 0.23
Lifetime alcohol use disorder 27 (79.4) 25 (78.1) 52 (78.8) 0.02
Anti Social Personality Disorder 5 (14.7) 2 (6.3) 7 (10.6) 1.24
Use a computer regularly 24 (70.6) 21 (65.6) 45 (68.2) 0.19
Continuous variables Mean (sd) Mean (sd) Mean (sd) F
Age 43.4 (9.6) 41.2 (9.7) 42.3 (9.6) 0.83
Days paid for work in past 28 3.2 (6.0) 2.9 (6.9) 3.1 (6.4) 0.04
Days of cocaine use past 28 15.2 (9.2) 14.5 (9.1) 14.8 (9.1) 0.09
Days of heroin use past 28 1.6 (4.9) 2.6 (6.4) 2.1 (5.6) 0.59
Days of alcohol use, past 28 0.4 (0.5) 0.3 (0.5) 0.4 (0.5) 0.64
Age of first cocaine use 20.6 (5.2) 19.5 (4.8) 20.1 (5.0) 0.87
Years of regular cocaine use 11.9 (6.9) 10.5 (8.7) 11.2 (7.8) 0.47
Lifetime number of arrests 13.5 (15.4) 11.6 (17.7) 12.5 (16.4) 0.22
# Previous Outpatient Treatments 3.5 (4.1) 2.1 (2.1) 2.9 (3.4) 2.95
# Previous Inpatient Treatments 4.4 (7.2) 2.7 (3.3) 3.6 (5.7) 1.38
Open in a new tab
*

all p values non-significant (>.05)

There were no significant differences in terms of treatment utilization in the main trial for those assigned to CBT4CBT compared to TAU within this subsample of participants included in the current study. The number of individual treatment sessions attended were comparable across conditions (CBT4CBT: M = 3.6, sd = 2.0; TAU: M = 4.2, sd = 3.4; F(1,55) = 0.51, p = ns), as were the number of group treatment sessions attended (CBT4CBT: M = 7.1, sd = 9.6; TAU: M = 9.2, sd = 16.8; F(1,55) = 0.32, p = ns). Those assigned to the CBT4CBT condition also completed an average of 5.5 (sd = 2.3) modules in the program. Also, there were no differences in terms of treatment completion within this subsample, as 82% of those assigned to CBT4CBT and 84% of those assigned to TAU completed treatment. These rates were comparable to those in the main study report (Carroll, Kiluk, Nich, Gordon, et al., 2014).

Of the 34 participants who completed the WAI-Tech at session 2, 27 (79%) completed it again at session 4, and 21 (62%) completed it again at session 8. Of the 32 participants assigned to TAU who completed the WAI at session 2, all (100%) completed it again at session 4, and 29 (91%) completed it again at session 8. Scores on the WAI-Tech or WAI at session 2 or 4 did not significantly differ between treatment completers or non-completers (no data was available for WAI-Tech or WAI at session 8 for treatment non-completers). There were also no significant differences in responses on the treatment evaluation measure between treatment completers and non-completers.

Psychometric evaluation of the WAI-Tech

Internal consistency of the WAI-Tech Total scale at session 2 (n = 34) was excellent (Cronbach’s α= .92), with each of the subscales considered to have good internal consistency (Task α= .84; Bond α= .78; Goal α= .75). These were comparable to Cronbach’s alpha coefficients for the CBT4CBT participants’ WAI Total scale at session 2 (n = 34; α= .94), and subscales (Task α= .86; Bond α= .89; Goal α= .86), as well as the TAU participants’ WAI Total scale at session 2 (n = 31; α= .95), and subscales (Task α= .86; Bond α= .85; Goal α= .86). In terms of the strength of relationship between the WAI-Tech and WAI total and corresponding subscales within the CBT4CBT sample (n = 34), there were strong positive correlations at session 2 (r ranged from .52 to .65, p < .01), with the lowest magnitude between the corresponding bond subscales (r = .52). At session 4, the total scale and subscales scores between WAI and WAI-Tech remained significant, although slightly lower magnitude (r ranged from .48 to .53, p < .01), however the bond subscales of these two measures were no longer correlated at session 4 (r = .28, p = ns). At session 8, again the bond subscales were not correlated (r = .23, p = ns), whereas the other subscales and total scale were positively correlated (r ranged from .51 to .63, p < .01).

Examination of descriptive statistics and frequencies for each of the WAI-Tech item ratings revealed that most clients viewed the CBT4CBT program positively. The negatively framed items had the lowest mean ratings, suggesting disagreement with a negative view of the program (e.g., item 7 – “I find what I am doing in these session confusing”: M = 1.56, sd = 0.79), whereas the positively framed items had the highest mean ratings (e.g., item 26 – “I trust what the CBT4CBT program is doing for me”: M = 5.56, sd = 1.08). Correlations of WAI-Tech scores and some of the additional items reflecting clients’ impression of the CBT4CBT program were consistent, indicating reasonable construct validity. For instance, the item “I can really relate to the characters in this program” was positively correlated with WAI-Tech scores (r ranged from .61 to .76, p < .001). The item, “I like the narrator in this computer program” was positively correlated with WAI-Tech bond subscale (r = .47, p < .01), more so than the task (r = .34, p < .05) or goal (r = .29, p = ns). Negatively framed items had a strong negative correlation with WAI-Tech scores. For example, the item “I find this computer program boring” was negatively correlated with the WAI-Tech scores (r ranged from −.54 to −.68, p < .001).

Participants’ rating of satisfaction with their substance use counselor was correlated with WAI-Tech total, task, and goal scores at session 4 (r ranged from .42 to .47, p < .05), and the task subscale at session 8 (r = .44, p < .05). This indicates that higher alliance scores on the WAI-Tech at those time points were associated with higher ratings of clients’ satisfaction with their counselor. However, the WAI-Tech bond subscale was not significantly related to participants’ satisfaction with their counselor at any time point. Clients’ ratings of alliance with their counselor on the WAI (for those assigned to CBT4CBT) were positively correlated with their ratings of satisfaction with their counselor, but only early in treatment (session 2 total, task, goal, and bond r ranged from .54 to .59, p < .01). Session 8 alliance ratings were not correlated with ratings of satisfaction with a counselor. This pattern was similar to that for clients assigned to TAU – earlier alliance ratings (session 2 and 4) were positively correlated with client satisfaction with their counselor (ranging from r = .51, p<.01, to r = .38, p<.05). Neither versions of the WAI were correlated with clients’ rating of satisfaction with treatment.

Stability of scores over time

Mean scores for the Total scale and subscales at each time point for both the WAI-Tech and WAI (CBT4CBT group only) are presented in Table 2. Overall, scores for the Total scale and subscale were not significantly different across the WAI versions, with the exception of the bond subscale at session 4 [F(1,52) = 5.78, p<.05]. Scores on the WAI-Tech bond subscale were lower than the WAI at all time points, however only the session 4 difference was statistically significant (WAI-Tech: M = 5.15, SD = 0.76; WAI: M = 5.71, SD = 0.92). There were no differences between WAI total or subscale scores across treatment conditions (CBT4CBT vs. TAU; data not displayed) at any time point, suggesting that inclusion of the computerized CBT component of the treatment did not significantly adversely affect the relationship with the counselor.

Table 2.

WAI-Tech and WAI mean scores over time (CBT4CBT group only)

WAI-Tech
(n= 34)		 WAI
(n= 34)
Session 2 mean std mean std F
task 5.75 0.82 5.65 1.08 0.21
bond 5.37 0.78 5.60 1.30 0.78
goal 5.52 0.75 5.64 1.03 0.30
total 5.55 0.70 5.63 1.08 0.14
Session 4 (n= 27) (n= 27)
task 5.64 0.79 5.69 0.81 0.06
bond 5.15 0.76 5.71 0.92 5.78*
goal 5.55 0.69 5.68 0.78 0.42
total 5.45 0.67 5.69 0.76 1.56
Session 8 (n= 21) (n= 21)
task 5.83 0.88 5.79 0.63 0.05
bond 5.09 0.93 5.51 1.08 1.87
goal 5.71 0.80 5.69 0.64 0.01
total 5.55 0.82 5.66 0.67 0.25
Open in a new tab
*

p < .05

Random effects regression evaluating change over time on the subscale mean scores between the WAI-Tech and WAI using all available data for those assigned to CBT4CBT, revealed a trend toward a time effect for the total score [F (1,136) = 2.92, p < .10], but no group or group by time interaction. The total scores appeared to decrease slightly over time for both WAI versions, however this seems to be due to the decrease in bond subscale scores over time rather than changes on the task or goal subscales. On the bond subscale, results indicated a significant time effect [F (1,138) = 6.79, p < .01] and a trend toward a group effect [F (1,129) = 3.12, p < .10], yet no group by time interaction. Specifically, the bond subscale scores were slightly lower overall on the WAI-Tech compared to the WAI, and also significantly decreased over time for both versions, but they did not differentially change over time. There were no group, time, or group by time interactions for the task or goal subscales, suggesting relatively stable scores on these subscales for both the WAI and WAI-Tech over time.

Relationships of WAI-Tech to Treatment Process and Outcomes

Correlations between the scales of the WAI versions at session 2 and several cocaine treatment outcome measures are displayed in Table 3. There were few significant relationships between the WAI-Tech and treatment outcome measures: the goal subscale was positively correlated with the percentage of days abstinent from cocaine (r = .41, p < .05), indicating higher ratings of perceived common goals with the computer program was associated with a greater percentage of days abstinent from cocaine during the treatment period. Because there were several significant relationships between baseline cocaine use characteristics (e.g., days of cocaine use prior to treatment entry) and treatment outcome measures (Carroll, Kiluk, Nich, Gordon, et al., 2014), the relationship between WAI-Tech scores at session 2 and percentage of cocaine use during treatment was no longer significant after controlling for these characteristics in multiple regression. Also, Structural Equation Modeling (SEM) to evaluate the association between WAI-Tech scores at the end of treatment (i.e., session 8) and cocaine use across the follow-up period (6-months following end of treatment) did not reveal any significant relationships.

Table 3.

Correlations of WAI at session 2 and treatment outcomes

WAI-Tech (Session 2)
(n = 34)		 CBT4CBT WAI (Session 2)
(n = 34)		 TAU WAI (Session 2)
(n = 32)
Outcome measure Task Bond Goal Total Task Bond Goal Total Task Bond Goal Total
Days in treatment .06 .20 .02 .11 .16 .29 .14 .22 −.24 −.19 −.21 −.22
% Cocaine negative urines −.02 .07 −.04 .01 .16 .27 .02 .17 .32 .28 .35* .33
% Days cocaine abstinent .30 .09 .41* .29 .24 .27 .15 .24 .40* .45** .39* .43*
≥ 21 days of consecutive abstinence .06 .06 .01 .05 .08 .24 −.10 .09 .23 .17 .23 .22
Open in a new tab
*

p < .05

**

p < .01

This pattern of relationships with treatment outcomes was similar to the ones found for the WAI among participants assigned to the CBT4CBT condition - the WAI in this group of participants had few relationships with treatment outcomes. However, among participants assigned to TAU, the WAI had several significant relationships with cocaine use outcomes. For instance, the WAI was positively associated with the percentage of days abstinent from cocaine use during treatment [total score (r = .43, p < .05), task (r = .40, p < .05), bond (r = .45, p < .01), goal (r = .39, p < .05)]. These positive correlations indicate higher client ratings of the working alliance with their counselor were associated with a greater percentage of days abstinent from cocaine during the treatment period.

In terms of treatment process, the goal and bond subscales for the WAI-Tech at session 4 were positively correlated with the number of CBT4CBT modules completed (r = .39, p < .05, and r = .38, p < .05, respectively), indicating a greater goal and bond alliance was associated with the completion of more CBT4CBT modules. WAI-Tech scores were not significantly correlated with the number of individual sessions with a substance use counselor during the study. Also, although WAI-Tech scores were not related to the participants’ rating of their satisfaction with the treatment received, they were related to the participants’ attribution of change. For instance, higher participant ratings on the item “In your opinion, do you believe that whether you’re worse, unchanged, or better (compared to when you began treatment) is related to the treatment received?” were associated with higher scores on the WAI-Tech at session 4: [total score (r = .39, p < .05), bond (r = .47, p < .05), goal (r = .48, p < .05)], and at session 8: [total score (r = .55, p < .01), task (r = .50, p < .05), bond (r = .61, p < .01), goal (r = .42, p < .05)]. Participant ratings on this item were not correlated with the WAI for those assigned to CBT4CBT, however they were correlated with the WAI for those assigned to TAU: [session 2 total score (r = .34, p < .05), bond (r = .36, p < .05), task (r = .41, p < .05)], and [session 8 total score (r = .51, p < .01), bond (r = .56, p < .01), goal (r = .45, p < .05), task (r = .49, p < .01)].

DISCUSSION

This study explored the level of working alliance between clients and technology-based interventions using a newly adapted version of the Working Alliance Inventory for Technology-based interventions (WAI-Tech) administered to cocaine dependent individuals enrolled in a randomized trial of CBT4CBT (Carroll, Kiluk, Nich, Gordon, et al., 2014). Overall, the WAI-Tech appeared to have similar psychometric characteristics as the standard WAI in terms of internal consistency, mean scores, and stability over time. Not surprisingly, the ‘bond’ subscale of the WAI-Tech was consistently lower than the other subscales and decreased over time, although this score also decreased over time for the WAI as well. Clients’ alliance with the computer program did not negatively affect the alliance with their counselor, as some higher WAI-Tech scores were associated with higher ratings of satisfaction with counselor. The most noteworthy finding was that scores on the WAI-Tech had no significant relationship with cocaine use treatment outcomes, whereas scores on the WAI for those not receiving the computerized intervention were consistently related to the percentage of days abstinent from cocaine.

This is one of the first evaluations of a working alliance assessment developed specifically to measure the relationship between the client and a technology-based intervention. Others have measured the alliance with a therapist/clinician who provided online counseling (Andersson et al., 2012; Holmes & Foster, 2012; Sucala et al., 2012), or provided brief check-ins in accordance with a computer-assisted treatment (e.g., Kay-Lambkin et al., 2011). Those that have measured a working alliance with a specific technology-based intervention did not include an evaluation of the psychometric characteristics of the adapted measure (Bickmore, Gruber, & Picard, 2005; Ormrod et al., 2010). Our adaptation of the established WAI (Horvath & Greenberg, 1989) was designed to measure the nature of the alliance that clients may form with a computer-based program, rather than with a live person, and was not to be confused with thoughts/feelings regarding the computer itself. This is a complex construct, as many technology-based interventions include no therapeutic interaction with a therapist/counselor, potentially removing this consistently strong treatment component from the intervention. Regarding the questions of the extent to which clients may form an alliance with a computer-based CBT program, and whether this alliance contributes to outcomes in the same manner as with a therapist-delivered psychotherapy, our preliminary findings are mixed.

Overall, results indicated the WAI-Tech appeared to retain the psychometric characteristics of the WAI, with the exception of the bond subscale. Internal consistency estimates were lower for this subscale on the WAI-Tech (although still within an acceptable range). In terms of relationships with clients’ rating of satisfaction with their counselor, scores on the WAI-Tech were largely not correlated (with the exception of the task, goal, and total scores following session 4). Clients’ levels of alliance with their counselor (as measured on WAI) were positively correlated with their ratings of satisfaction with their counselor, as expected. Taken together, this suggests that clients’ satisfaction with their counselor is not negatively impacted by whether they are also receiving a computer-based treatment in addition to standard outpatient counseling, nor by any potential alliance with the computer-based treatment. Interestingly, clients’ levels of alliance with either the computer-based CBT program, or their counselor, was not related to their ratings of satisfaction with treatment. This was true across treatment conditions for reasons that are unclear.

When comparing scores across WAI versions, the bond subscale of the WAI-Tech continued to be the outlier relative to the task and bond scales. Mean scores were consistently lower for the bond subscale on the WAI-Tech compared to the WAI, but differences were only significant at session 4. The other subscale scores were not significantly different across versions at each time point. This makes intuitive sense, as the ‘bond’ concept of the working alliance refers to the personal attachment between client and therapist, and includes issues such as mutual trust, acceptance, and confidence, whereas the ‘tasks’ concept refers to perception of therapeutic tasks as relevant and efficacious, and the ‘goals’ concept refers to the agreement regarding the outcome of the intervention (Bordin, 1979). Thus, clients may agree with the relevance and outcomes of the therapeutic tasks within a technology-delivered intervention as much as when delivered by a therapist, yet they may not develop the same level of trust and acceptance with a computer/technology-based program as they would with a therapist in-person or online.

However, the clients’ ratings of perceived bond with CBT4CBT weren’t particularly low, per se; mean ratings were above the neutral midpoint at all time points indicating a positive alliance, consistent with other reports of alliance with computerized CBT (Ormrod et al., 2010). Ratings of the bond alliance may depend in part on the ‘relational’ aspects of the technology-based intervention. Bickmore, Gruber, and Picard (2005) found higher ratings on bond subscales of an adapted WAI, with increases over time, when a technology-based intervention for exercise adoption included a ‘relational agent’ – a computer animated human that interacted with users in a simulated face-to-face conversation using both verbal (e.g., empathy, social dialogue, humor, etc.) and nonverbal behaviors (e.g., direct gaze, smiling, nodding, etc.) to establish and maintain a working alliance, compared to an intervention with a ‘non-relational agent’ - the same computer animated human who provided exercise advice without these more complex verbal and nonverbal behaviors. The CBT4CBT program in the current study included a human narrator who acted as a guide throughout the program (yet was only visible on some screen pages). This may have contributed to the reasonably high bond subscale scores in this sample, which were actually higher than the bond subscales scores reported by Bickmore, Gruber, and Picard (2005) for participants receiving a technology-based intervention with a ‘relational agent’. Moreover, strong correlations between alliance ratings on the WAI-Tech and items indicating a positive impression of the narrator and characters in the computer-based CBT provide some support for the potential significance of ‘relational’ aspects of technology-based interventions for development of an alliance.

These data also suggest that WAI-Tech ratings were not strongly related to cocaine use outcomes during the treatment or follow-up period, which is consistent with reports in other areas indicating no relationship between treatment outcomes and alliance within technology-based interventions (Andersson et al., 2012; Bickmore, Gruber, & Picard, 2005; Knaevelsrud & Maercker, 2006; Ormrod et al., 2010; Preschl et al., 2011). Conversely, for those assigned to TAU, higher client-reported alliance with their therapist at several time points during the treatment period was associated with greater rates of abstinence from cocaine (i.e., greater percentage of days abstinent). This is consistent with most prior studies of the working alliance and outcome in substance abuse treatment (Joe, Simpson, Dansereau, & Rowan-Szal, 2001; Simpson, Joe, Rowan-Szal, & Greener, 1997) However, this was not true for the client-reported alliance with their therapist among participants in the CBT4CBT condition (who received both a therapist- and technology-delivered intervention). In other words, a stronger alliance with a therapist/counselor was related to less cocaine use during treatment, but not for those who also received the computer-based program in addition to treatment from their therapist. Moreover, a higher client-rated attribution that change was due to treatment received was associated with higher alliance scores on the WAI-Tech but not the WAI for those in the CBT4CBT condition (there was an association with the WAI for those receiving TAU only). Therefore, while the level of alliance with a technology-based intervention may be related to whether clients attribute change to the treatment received, it does not appear related to substance use the same way the level of alliance with a therapist/counselor in a face-to-face treatment is related.

Such interpretations should be approached with caution, given the number of limitations in this study. First, this preliminary evaluation of the WAI-Tech was limited by the small sample size, which precluded more advanced evaluation of the measure, such as factor analysis and/or item analysis. Also, the reduced sample of participants completing the WAI-Tech at all three time points further limited the potential analyses. That being said, correlations with the WAI and other indicators of satisfaction with the CBT4CBT program lend some reasonable construct validity, and the use of random effects regression analyses for evaluating scores over time minimized the effect of missing data. Second, the multiple correlation analyses conducted in this study potentially increased the likelihood of chance findings, which may suggest the need for a more conservative significance level to account for inflated Type I error rates. Using a significance level of p<.01, for instance, would slightly reduce the number of relationships interpreted as significant. However, the small sample size and moderate strength of the correlation coefficient values mitigate the potential for false positives at p<.05. Third, this study did not obtain WAI ratings from the clinicians, limiting the ability to separate the variability in WAI scores according to therapist variability and client variability (Baldwin et al., 2007; Crits-Christoph et al., 2009). Last, the sample in this study was drawn from a methadone-maintained population which limits the generalizability of the findings to other populations of substance users and other psychiatric disorders in general.

Taken together, the results of this study suggest that some aspects of the construct of a working alliance may apply to a computer-based CBT program, with clients forming a similar level of alliance with the goals and tasks of the computer program as they would with a live therapist. However, the nature of the ‘bond’ between a client and a technology-based intervention is unclear. Clients’ level of alliance with a technology-based intervention does not appear to affect their level of alliance with a therapist when a treatment includes both delivery formats, yet the alliance with a technology-based intervention may not have the same impact on treatment outcomes as it does in a therapist-delivered intervention. Therefore, the distinctive treatment ingredients (i.e., specific factors) of the intervention may be a more important contributor to outcomes for technology-based interventions, whereas the common treatment ingredients (i.e., therapist behaviors) may play a larger role in face-to-face therapeutic interventions. Future research should continue to explore the role of a working alliance for technology-based interventions that do not include a therapist component.

HIGHLIGHTS.

  • We adapted the Working Alliance Inventory for Technology-based Interventions.

  • We examined WAI-Tech within randomized trial of CBT4CBT for cocaine use.

  • Levels of alliance with CBT4CBT program were similar to alliance with therapist.

  • Alliance with computer-based program did not affect alliance with therapist.

  • Alliance with computer-based program not associated with treatment outcomes.

ACKNOWLEDGEMENTS

This research was supported in part through National Institute on Drug Abuse (NIDA) grants R01DA015969 and P50DA09241(PI: Carroll). The funding source had no other role than financial support. We wish to acknowledge Carly J. Gibbons for her role in developing the WAI-Tech, and Karen Hunkele for her role as a data manager on the CBT4CBT randomized trial.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

REFERENCES

  1. Agnew-Davies R, Stiles WB, Hardy GE, Barkham M, Shapiro DA. Alliance structure assessed by the Agnew Relationship Measure (ARM) Br J Clin Psychol. 1998;37(Pt 2):155–172. doi: 10.1111/j.2044-8260.1998.tb01291.x. [DOI] [PubMed] [Google Scholar]
  2. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. Fourth Edition. Washington DC: APA Press; 1994. [Google Scholar]
  3. Andersson G, Paxling B, Wiwe M, Vernmark K, Felix CB, Lundborg L, Carlbring P. Therapeutic alliance in guided internet-delivered cognitive behavioural treatment of depression, generalized anxiety disorder and social anxiety disorder. Behaviour Research and Therapy. 2012;50(9):544–550. doi: 10.1016/j.brat.2012.05.003. doi: http://dx.doi.org/10.1016/j.brat.2012.05.003; [DOI] [PubMed] [Google Scholar]
  4. Baldwin SA, Wampold BE, Imel ZE. Untangling the alliance-outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance. Journal of Consulting and Clinical Psychology. 2007;75(6):842–852. doi: 10.1037/0022-006X.75.6.842. [DOI] [PubMed] [Google Scholar]
  5. Barber JP, Connolly MB, Crits-Christoph P, Gladis L, Siqueland L. Alliance predicts patients' outcome beyond in-treatment change in symptoms. J Consult Clin Psychol. 2000;68(6):1027–1032. doi: 10.1037//0022-006x.68.6.1027. [DOI] [PubMed] [Google Scholar]
  6. Bickel WK, Marsch LA, Buchhalter AR, Badger GJ. Computerized behavior therapy for opioid-dependent outpatients: a randomized controlled trial. Exp Clin Psychopharmacol. 2008;16(2):132–143. doi: 10.1037/1064-1297.16.2.132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bickmore T, Gruber A, Picard R. Establishing the computer-patient working alliance in automated health behavior change interventions. Patient Educ Couns. 2005;59(1):21–30. doi: 10.1016/j.pec.2004.09.008. [DOI] [PubMed] [Google Scholar]
  8. Bickmore Timothy W, Caruso Lisa, Clough-Gorr Kerri, Heeren Tim. ‘It's just like you talk to a friend’ relational agents for older adults. Interacting with Computers. 2005;17(6):711–735. doi: http://dx.doi.org/10.1016/j.intcom.2005.09.002; [Google Scholar]
  9. Bordin Edward S. The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research & Practice. 1979;16(3):252–260. [Google Scholar]
  10. Budney Alan J, Fearer Stephanie, Walker Denise D, Stanger Catherine, Thostenson Jeff, Grabinski Michael, Bickel Warren K. An initial trial of a computerized behavioral intervention for cannabis use disorder. Drug & Alcohol Dependence. 2011;115(1/2):74–79. doi: 10.1016/j.drugalcdep.2010.10.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Carroll KM. A Cognitive-Behavioral Approach: Treating Cocaine Addiction. Rockville, Maryland: NIDA; 1998. [Google Scholar]
  12. Carroll KM, Ball SA, Martino S, Nich C, Babuscio T, Gordon MA, Rounsaville BJ. Computer-assisted cognitive-behavioral therapy for addiction. A randomized clinical trial of 'CBT4CBT'. American Journal of Psychiatry. 2008;165(7):881–888. doi: 10.1176/appi.ajp.2008.07111835. PMC2562873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Carroll KM, Fenton LR, Ball SA, Nich C, Frankforter TL, Shi J, Rounsaville BJ. Efficacy of disulfiram and cognitive-behavioral therapy in cocaine-dependent outpatients: A randomized placebo controlled trial. Archives of General Psychiatry. 2004;64:264–272. doi: 10.1001/archpsyc.61.3.264. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Carroll KM, Kiluk BD, Nich C, DeVito EE, Decker S, LaPaglia D, Ball SA. Towards empirical identification of a clinically meaningful indicator of treatment outcome for drug addiction: Features of candidate indicators and evaluation of sensitivity to treatment effects and relationship to one year cocaine use follow-up outcomes. Drug and Alcohol Dependence. 2014;137:3–19. doi: 10.1016/j.drugalcdep.2014.01.012. PMC in progress. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Carroll KM, Kiluk BD, Nich C, Gordon MA, Portnoy G, Marino D, Ball SA. Computer-assisted delivery of cognitive-behavioral therapy: Efficacy and durability of CBT4CBT among cocaine-dependent individuals maintained on methadone. American Journal of Psychiatry. 2014;171:436–444. doi: 10.1176/appi.ajp.2013.13070987. in progress. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Carroll KM, Rounsaville BJ. Computer-assisted therapy in psychiatry: Be brave - it’s a new world. Current Psychiatry Reports. 2010;12(5):426–432. doi: 10.1007/s11920-010-0146-2. PMC2967758. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Carroll Kathleen M, Ball Samuel A, Martino Steve, Nich Charla, Babuscio Theresa A, Rounsaville Bruce J. Enduring effects of a computer-assisted training program for cognitive behavioral therapy: A 6-month follow-up of CBT4CBT. Drug and Alcohol Dependence. 2009;100(1):178–181. doi: 10.1016/j.drugalcdep.2008.09.015. PMC2742309. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Cecero JJ, Fenton LR, Nich C, Frankforter T, Carroll KM. Focus on the therapeutic alliance: The psychometric properties of six measures across three treatments. Psychotherapy. 2001;38:1–11. [Google Scholar]
  19. Connors GJ, Carroll KM, DiClemente CC, Longabaugh R, Donovan DM. The therapeutic alliance and its relationship to alcoholism treatment participation and outcome. Journal of Consulting and Clinical Psychology. 1997;65:588–598. doi: 10.1037//0022-006x.65.4.588. [DOI] [PubMed] [Google Scholar]
  20. Connors GJ, DiClemente CC, Derman KH, Kadden RM, Carroll KM. Predicting the therapeutic alliance in alcoholism treatment. Journal of Studies on Alcohol. 2000;61:139–149. doi: 10.15288/jsa.2000.61.139. [DOI] [PubMed] [Google Scholar]
  21. Cook JE, Doyle C. Working alliance in online therapy as compared to face-to-face therapy: Preliminary results. Cyberpsychology and Behavior. 2002;5(2):95–105. doi: 10.1089/109493102753770480. [DOI] [PubMed] [Google Scholar]
  22. Crits-Christoph P, Baranackie K, Kurcias J, Beck AT, Carroll KM, Perry K, Zitrin C. Meta-analysis of therapist effects in psychotherapy outcome studies. Psychotherapy Research. 1991;1:81–91. [Google Scholar]
  23. Crits-Christoph P, Gallop R, Temes CM, Woody G, Ball SA, Martino S, Carroll KM. The alliance in motivational enhancement therapy and counseling as usual for substance abuse problems. Journal of Consulting & Clinical Psychology. 2009;77(6):1125–1135. doi: 10.1037/a0017045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Cuijpers Pim, Marks Isaacx M, van Straten Annemieke, Cavanagh Kate, Gega Lina, Andersson Gerhard. Computer-aided psychotherapy for anxiety disorders: A meta-analytic review. Cognitive Behaviour Therapy. 2009;38(2):66–82. doi: 10.1080/16506070802694776. [DOI] [PubMed] [Google Scholar]
  25. Donovan Dennis M, Bigelow George E, Brigham Gregory S, Carroll Kathleen M, Cohen Allan J, Gardin John G, Wells Elizabeth A. Primary outcome indices in illicit drug dependence treatment research: Systematic approach to selection and measurement of drug use endpoints in clinical trials. Addiction. 2012;107(4):694–708. doi: 10.1111/j.1360-0443.2011.03473.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Elkin Irene, Falconnier Lydia, Martinovich Zoran, Mahoney Colleen. Therapist effects in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychotherapy Research. 2006;16(2):144–160. [Google Scholar]
  27. Falkenstrom F, Granstrom F, Holmqvist R. Working alliance predicts psychotherapy outcome even while controlling for prior symptom improvement. Psychother Res. 2014;24(2):146–159. doi: 10.1080/10503307.2013.847985. [DOI] [PubMed] [Google Scholar]
  28. Fals-Stewart W, O'Farrell TJ, Freitas TT, McFarlin SK, Rutigliano P. The timeline followback reports of psychoactive substance use by drug-abusing patients: Psychometric properties. Journal of Consulting and Clinical Psychology. 2000;68:134–144. doi: 10.1037//0022-006x.68.1.134. [DOI] [PubMed] [Google Scholar]
  29. Fenton LR, Cecero JJ, Nich C, Frankforter TL, Carroll KM. Perspective is everything: The predictive validity of six working alliance instruments. Journal of Psychotherapy Practice and Research. 2001;10:262–268. [PMC free article] [PubMed] [Google Scholar]
  30. Gibbons CJ, Nich C, Steinberg K, Roffman RA, Corvino J, Babor TF, Carroll KM. Treatment process, alliance and outcome in brief versus extended treatments for marijuana dependence. Addiction. 2010;105(10):1799–1808. doi: 10.1111/j.1360-0443.2010.03047.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Gustafson David H, Boyle Michael G, Shaw Bret R, Isham Andrew, McTavish Fiona, Richards Stephanie, Johnson Kim. An e-health solution for people with alcohol problems. Alcohol Research & Health. 2011;33(4):327–337. [PMC free article] [PubMed] [Google Scholar]
  32. Hester Reid K, Delaney Harold D, Campbell William. ModerateDrinking.Com and moderation management: Outcomes of a randomized clinical trial with non-dependent problem drinkers. Journal of Consulting and Clinical Psychology. 2011;79(2):215–224. doi: 10.1037/a0022487. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Holmes C, Foster V. A preliminary comparison study of online and face-to-face counseling: Client perceptions of three factors. Journal of Technology in Human Services. 2012;30(1):14–31. [Google Scholar]
  34. Horvath AO, Del Re AC, Flückiger C, Symonds D. Alliance in individual psychotherapy. Psychotherapy. 2011;48(1):9–16. doi: 10.1037/a0022186. [DOI] [PubMed] [Google Scholar]
  35. Horvath AO, Greenberg LS. The development of the Working Alliance Inventory. In: Greenberg LS, Pinsof WM, editors. The Psychotherapeutic Process: A Research Handbook. New York: Guilford; 1986. pp. 529–556. [Google Scholar]
  36. Horvath AO, Greenberg LS. Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology. 1989;36(2):223–233. [Google Scholar]
  37. Horvath AO, Symonds BD. Relation between working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology. 1991;38:139–149. [Google Scholar]
  38. Ilgen MA, McKellar J, Moos RH, Finney JW. Therapeutic alliance and the relationship between motivation and treatment outcomes in patients with alcohol use disorder. Journal of Substance Abuse Treatment. 2006;31(2):157–162. doi: 10.1016/j.jsat.2006.04.001. doi: http://dx.doi.org/10.1016/j.jsat.2006.04.001; [DOI] [PubMed] [Google Scholar]
  39. Joe GW, Simpson DD, Dansereau DF, Rowan-Szal GA. Relationships between counseling rapport and drug abuse treatment outcomes. Psychiatric Services. 2001;52(9):1223–1229. doi: 10.1176/appi.ps.52.9.1223. [DOI] [PubMed] [Google Scholar]
  40. Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, Sutcliffe P. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health Technol Assess. 2006;10(33):iii, xi–xiv, 1–168. doi: 10.3310/hta10330. [DOI] [PubMed] [Google Scholar]
  41. Kaplan B, Farzanfar R, Friedman RH. Personal relationships with an intelligent interactive telephone health behavior advisor system: a multimethod study using surveys and ethnographic interviews. Int J Med Inform. 2003;71(1):33–41. doi: 10.1016/s1386-5056(03)00072-8. [DOI] [PubMed] [Google Scholar]
  42. Kay-Lambkin F, Baker A, Lewin T, Carr V. Acceptability of a clinician-assisted computerized psychological intervention for comorbid mental health and substance use problems: Treatment adherence data from a randomized controlled trial. Journal of Medical Internet Research. 2011;13(1):339–349. doi: 10.2196/jmir.1522. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Kim Don-Min, Wampold Bruce E, Bolt Daniel M. Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health Treatment of Depression Collaborative Research Program data. Psychotherapy Research. 2006;16(2):161–172. [Google Scholar]
  44. Knaevelsrud C, Maercker A. Does the quality of the working alliance predict treatment outcome in online psychotherapy for traumatized patients? J Med Internet Res. 2006;8(4):e31. doi: 10.2196/jmir.8.4.e31. PMC1794007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Knaevelsrud C, Maercker A. Internet-based treatment for PTSD reduces distress and facilitates the development of a strong therapeutic alliance: a randomized controlled clinical trial. BMC Psychiatry. 2007;7:13. doi: 10.1186/1471-244X-7-13. PMC1885249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Marks I, Cavanagh K. Computer-aided psychological treatments: Evolving issues. Annual Review of Clinical Psychology. 2009;5:121–141. doi: 10.1146/annurev.clinpsy.032408.153538. [DOI] [PubMed] [Google Scholar]
  47. Marks Isaac M, Cuijpers Pim, Cavanagh Kate, van Straten Annemieke, Gega Lina, Andersson Gerhard. Meta-analysis of computer-aided psychotherapy: Problems and partial solutions. Cognitive Behaviour Therapy. 2009;38(2):83–90. doi: 10.1080/16506070802675239. [DOI] [PubMed] [Google Scholar]
  48. Marsch LA, Dallery J. Advances in the psychosocial treatment of addiction: the role of technology in the delivery of evidence-based psychosocial treatment. Psychiatr Clin North Am. 2012;35(2):481–493. doi: 10.1016/j.psc.2012.03.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Martin DJ, Garske JP, Davis MK. Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology. 2000;68:438–450. [PubMed] [Google Scholar]
  50. Meier PS, Barrowclough C, Donmall MC. The role of the therapeutic alliance in the treatment of substance misuse: a critical review of the literature. Addiction. 2005;100(3):304–316. doi: 10.1111/j.1360-0443.2004.00935.x. [DOI] [PubMed] [Google Scholar]
  51. Meier PS, Donmall MC, McElduff P, Barrowclough C, Heller RF. The role of the early therapeutic alliance in predicting drug treatment dropout. Drug and Alcohol Dependence. 2006;83(1):57–64. doi: 10.1016/j.drugalcdep.2005.10.010. doi: http://dx.doi.org/10.1016/j.drugalcdep.2005.10.010; [DOI] [PubMed] [Google Scholar]
  52. Ondersma SJ, Chase SK, Svikis DS, Schuster CR. Computer-based brief motivational intervention for perinatal drug use. J Subst Abuse Treat. 2005;28(4):305–312. doi: 10.1016/j.jsat.2005.02.004. doi: S0740-5472(05)00044-9 [pii] 10.1016/j.jsat.2005.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Ormrod JA, Kennedy L, Scott J, Cavanagh K. Computerised cognitive behavioural therapy in an adult mental health service: a pilot study of outcomes and alliance. Cogn Behav Ther. 2010;39(3):188–192. doi: 10.1080/16506071003675614. [DOI] [PubMed] [Google Scholar]
  54. Preschl B, Maercker A, Wagner B. The working alliance in a randomized controlled trial comparing online with face-to-face cognitive-behavioral therapy for depression. BMC Psychiatry. 2011;11:189. doi: 10.1186/1471-244X-11-189. PMC3248847. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Richardson Thomas, Stallard Paul, Velleman Sophie. Computerised cognitive behavioural therapy for the prevention and treatment of depression and anxiety in children and adolescents: A systematic review. Clinical Child and Family Psychology Review. 2010;13(3):275–290. doi: 10.1007/s10567-010-0069-9. [DOI] [PubMed] [Google Scholar]
  56. Simpson DD, Joe GW, Rowan-Szal GA, Greener JM. Drug abuse treatment process components that improve retention. Journal of Substance Abuse Treatment. 1997;14(6):565–572. doi: 10.1016/s0740-5472(97)00181-5. [DOI] [PubMed] [Google Scholar]
  57. Sobell LC, Sobell MB. Timeline followback: A technique for assessing self-reported alcohol consumption. In: Litten RZ, Allen J, editors. Measuring alcohol consumption: Psychosocial and biological methods. New Jersey: Humana Press; 1992. pp. 41–72. [Google Scholar]
  58. Spek V, Cuijpers P, Nyklicek I, Riper H, Keyzer J, Pop V. Internet-based cognitive-behaviour therapy for symptoms of depression and anxiety: A meta-analysis. Psychological Medicine. 2007;37:319–328. doi: 10.1017/S0033291706008944. [DOI] [PubMed] [Google Scholar]
  59. Strunk DR, Brotman MA, DeRubeis RJ. The process of change in cognitive therapy for depression: predictors of early inter-session symptom gains. Behav Res Ther. 2010;48(7):599–606. doi: 10.1016/j.brat.2010.03.011. Pmc2878902. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Sucala M, Schnur JB, Constantino MJ, Miller SJ, Brackman EH, Montgomery GH. The therapeutic relationship in e-therapy for mental health: A systematic review. Journal of Medical Internet Research. 2012;14(4):175–187. doi: 10.2196/jmir.2084. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Tichenor V, Hill CE. A comparison of six measures of the working alliance. Psychotherapy. 1989;26:195–199. [Google Scholar]
  62. Tracey TJ, Kokotovic AM. Factor structure of the Working Alliance Inventory. Psychological Assessment. 1989;1:207–210. [Google Scholar]
  63. Webb CA, DeRubeis RJ, Amsterdam JD, Shelton RC, Hollon SD, Dimidjian S. Two aspects of the therapeutic alliance: differential relations with depressive symptom change. J Consult Clin Psychol. 2011;79(3):279–283. doi: 10.1037/a0023252. Pmc3124012. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES