Skip to main content
Internet Interventions logoLink to Internet Interventions
. 2017 Nov 17;11:1–10. doi: 10.1016/j.invent.2017.11.005

Therapeutic alliance in guided internet therapy programs for depression and anxiety disorders – A systematic review

Satu Pihlaja 1, Jan-Henry Stenberg 1, Kaisla Joutsenniemi 1, Heidi Mehik 1, Ville Ritola 1, Grigori Joffe 1,
PMCID: PMC6084872  PMID: 30135754

Abstract

Objective

The role of internet therapy programs for mental disorders is growing. Those programs employing human support yield better outcomes than do those with no such support. Therapeutic alliance may be a critical element in this support. Currently, the significance of therapeutic alliance in guided, internet-delivered cognitive behavioral therapy programs (iCBT) remains unknown. This review aims to determine whether the therapeutic alliance influences outcome of iCBTs and if it does, what plausible factors underlie this association.

Method

Towards that goal searches were made in PubMed, PsycINFO, SCOPUS, The Cochrane Library and CINAHL in May 2016 and January 2017.

Results

From the 1658 relevant studies, only six studied the relationship of therapeutic alliance and outcome. All six studies showed a high level of client-therapist alliance; in the three most recent studies, the alliance was directly associated with outcome. No studies reported alliance-adherence associations.

Conclusions

Alliance research in iCBT for mental disorders is scarce. Therapeutic alliance seems to associate with outcomes. More studies are necessary to define the optimal support to strengthen alliance. iCBT is a feasible environment for alliance research both practically and theoretically. The impact of alliance on adherence to iCBT requires study.

Keywords: Therapeutic relationship, Alliance, Internet-based therapy, CBT, iCBT

Highlights

  • Therapeutic alliance in iCBT is high and may be even stronger than in face to face therapy.

  • Alliance appears to directly associate with clinical outcomes but the research is scarce.

  • Data on effects of alliance on adherence are missing.

  • iCBT is a plentiful environment for both practical and theoretical research of therapeutic alliance.

1. Introduction

Mental Health Disorders account for 28% of Days Lived with Disability (DALY)–measured global burden of disease among non-communicable diseases – more than cardiovascular diseases or cancer (WHO, 2011). Depressive disorders are the leading cause of disability throughout the world and contribute tremendously to the overall global burden of disease (WHO, 2017). Anxiety disorders are the sixth leading cause of disability worldwide (Baxter et al., 2014) and are, as well, a major component of the global burden of disease. Depression and anxiety disorders, the most prevalent mental health problems (Whiteford et al., 2015) show great comorbidity (Kaufman and Charney, 2000), phenomenological and genetic overlap (Hattema, 2008), similarities in pharmacological treatment (Levine et al., 2001) and appear to share the same background mechanisms (Rosellini and Brown, 2011).

For depressive and anxiety disorders, psychological interventions are among first-line treatments (McHugh et al., 2013) and cognitive behavior therapy (CBT) is recommended in many national treatment guidelines, e.g. those of National Institute for Health and Clinical Excellence (2013). Psychotherapies are also highly acceptable among clients (Leykin et al., 2007, van Schaik et al., 2004). Despite their acceptability and feasibility, psychotherapies, however, are not available for all those who could benefit from them (Kohn et al., 2004, Young et al., 2001). Obstacles to use include perceived stigma, shortage of professionals, costs, and long distances to services (Mechanic, 2007).

Research to date indicates that therapeutic internet-based interventions in treating depression and anxiety disorders are valuable (Andrews et al., 2010, Richards and Richardson, 2012, Saddichha et al., 2014). In addition, they offer solutions to problems of inequality, since they are affordable, lack location and time constraints, and offer a steady quality of treatment (Andersson et al., 2013, Andrews et al., 2010, Cuijpers et al., 2009). A distinction between various types of treatment delivery via the internet is necessary, since the nature of treatment is different in client-therapist videoconference from that in an asynchronous, computer-based therapy program with additional therapist guidance (Berger, 2015). Barak et al. (2009) defined the latter as “human supported, web-based therapeutic interventions” and Berger (2015) as “internet-based guided self-help treatments”.

Research in the field of such computer-assisted, internet-delivered asynchronous interventions focuses mostly on human-supported, internet-based cognitive behavioral therapy (from now on, iCBT) (Aboujaoude et al., 2015, Berger, 2015). These treatments outperform unsupported self-help computer-based programs (Baumeister et al., 2014) and appear to be as efficacious as (Berger, 2015), but substantially less resource-consuming (Andersson et al., 2013, Andersson and Cuijpers, 2009, Andersson et al., 2014, Andrews et al., 2010, Cuijpers et al., 2009, Barak et al., 2008, Richards and Richardson, 2012) than are traditional face-to-face therapies.

What exactly makes human support so important in iCBT programs is largely unknown.

Professionals have been concerned about a possible lack of therapeutic alliance with the supporting internet therapist in the iCBT (MacLeod et al., 2009, Sucala et al., 2012). Therapeutic alliance is defined as a positive emotional bond between therapist and client, and their mutual agreement on the goals and tasks of the treatment (Bordin, 1994). Alliance is important in predicting the outcome of traditional face-to-face psychotherapy (Norcross, 2011). Interestingly, alliance ratings in iCBTs have been as high (Sucala et al., 2012) or even higher than are those of the traditional face-to-face psychotherapies (Berger, 2015).

Descriptions of effective psychotherapies should always include consideration of therapeutic alliance (Ackerman et al., 2001). For theoretical reasons, it would be important to know whether alliance is a predictive or mediating factor also in the iCBT (Cavanagh and Millings, 2013). Moreover, exploration of the therapeutic relationship in the technological environment of iCBTs may foster better understanding of the nature of therapeutic relationship itself. In iCBT, the therapeutic alliance has been suggested to be less important than in traditional psychotherapy, since typically very little contact occurs between the client and therapist. Understanding the alliance is important for practical reasons as well. Specifically, that may be an issue of optimal resource allocation if the alliance influences treatment outcomes. If it does, suitable support should be available, for instance, to strengthen the alliance (Berger, 2015, Cavanagh and Millings, 2013). Studies on this matter have revealed mixed results (Andersson et al., 2012b, Knaevelsrud and Maercker, 2006). Associations between alliance ratings and treatment outcomes most often show positive trend but without always achieving statistical significance (Berger, 2015).

Even if therapeutic alliance does not directly predict treatment outcome in the iCBT, alliance-building may support adherence to treatment and thereby prevent premature discontinuation of that treatment (Hilvert-Bruce et al., 2012, Richards and Richardson, 2012). Adherence is typically defined as proportion of program completers. In this article also amount of treatment modules completed is considered as a measure of adherence. Premature discontinuation of iCBT involving minimal therapist contact ranged from two to 83% with a weighted average of 31% (Melville et al., 2010). Attrition in randomized controlled trials for depressive and anxiety disorders ranged from approximately 1–50% (Christensen et al., 2009). Adherence in iCBT efficacy trials for depression has been high, 75–85% (Hilvert-Bruce et al., 2012). Indeed, adherence to internet-based cognitive therapy treatments in terms of increased program exposure (Christensen et al., 2004) and complying to the therapeutic tasks (Simpson et al., 2011) are associated with successful clinical outcomes (Christensen et al., 2002, Hilvert-Bruce et al., 2012). Differences in the grade of therapeutic alliance might be an important determinant explaining the wide range of retention in iCBT treatments, but studies on the alliance-retention association are still rare.

Available reviews concerning alliance in internet interventions either focus on videoconferencing psychotherapy (Simpson and Reid, 2014), fail to differentiate between various types of interventions (Sucala et al., 2012), provide only narrative results (Berger, 2015) or include a wide range of psychological problems (Barazzone et al., 2012, Berger, 2015, Sucala et al., 2012), making between-study comparisons difficult. This review aims to find whether the therapeutic alliance influences outcome of and adherence to iCBTs, and if it does, what plausible factors underlie this association. This review is limited to individual iCBTs in adults with the most common mental disorders, specifically depressive and anxiety disorders. In order to find common elements of support, studies concerning disorders beyond depression and anxiety (and thus, less likely sharing the same background alliance-related mechanisms) or special populations like adolescents, psychotic and trauma-based populations (that may need specialized support) were excluded. Based on the literature, what is to be expected is that the therapeutic alliance is connected with treatment outcome, but associations may not reach statistical significance (Berger, 2015).

2. Methods

2.1. Literature search

The systematic database search and additional hand search took place in June 2016, and complementary searches in January 2017 in five databases (PubMed, PsycINFO, SCOPUS, Cochrane Library and CINAHL).

The search strategy used was (guided OR guidance OR support OR alliance) AND (computer-based psychotherapy OR web-based psychotherapy OR internet-based psychotherapy OR internet-based cognitive behavioral therapy OR self-help cognitive behavioral therapy) AND (internet). Detailed search histories are available from the first author. The hand search included searching the references of the studies found through the database search.

2.2. Selection of studies

Studies comprised individual iCBTs for different depressive and anxiety disorders. Exclusion criteria were group or family iCBTs, iCBTs for conditions requiring special or different support (like adolescents, psychotic and trauma-based populations). Client variables, dosage of support, modes of contact, and education of internet therapists were explored as plausible alliance-related variables.

The records were screened in the three phases described in Fig. 1. Inclusion criteria were: 1) internet-based psychotherapy intervention, 2) individual treatment, 3) guided interventions. 4) records written in English, 5) peer reviewed. First, 1654 abstracts emerged from the database search, and 4 abstracts through manual search. Exclusion criteria were: 6) virtual reality or game interventions, 7) study protocols alone, 8) only cost-effectiveness analysis 9). A total of 1159 articles were excluded. The 499 abstracts left were screened by the following additional inclusion/exclusion criteria: 10) interventions targeted at mental disorders, 11) intervention was not targeted at substance abuse or pathological gambling, psychotic or trauma-related disorders, clients in residential care, immigrant, religious, older-adults populations, students, school pupils, or clients aged under 18 years and 12) measured therapeutic alliance. These criteria excluded 489 articles.

Fig. 1.

Fig. 1

PRISMA 2009 flow diagram.

Of the remaining 10 articles, two concerned therapeutic alliance with the computer program rather than with the internet therapist (Berger et al., 2014, Clarke et al., 2016, 32,33), and four others (Hadjistavropoulos et al., 2016, Kiropoulos et al., 2008, Lindner et al., 2014, Reynolds et al., 2006) made no exploration of the association of therapeutic alliance with outcome. These six studies were, hence, excluded. Of the four remaining studies (Andersson et al., 2015, Andersson et al., 2012b, Bergman Nordgren et al., 2013, Herbst et al., 2016), one study (Andersson et al., 2012b) reported secondary alliance analysis of three samples previously studied, so altogether six trials reporting alliance ratings were included in the analysis. (Fig. 1).

2.3. Data extraction

The following data describing the trials were extracted from the studies independently by two assessors (SP, KJ): mental disorder studied, basic sample characteristics (sample size, mean age), outcome measures, number of modules and length of treatment, study design, and effect size of the treatment (Table 1). For exploring potentially alliance-related factors, the following characteristics were extracted: mean and percentage of completed modules, number and percentage of treatment completers, selection of sample population, therapist education and time required (per client), modality and frequency of contact, other reported contacts during treatment, alliance measure used, time-point of alliance measurement, alliance rating scores, and alliance effect on outcome.

Table 1.

Study characteristics.

Authors, astudied disorder Sample size and mean age bOutcome measures, main measures bolded Treatment length (weeks/modules) Study design, cRCT: diCBT vs. control Within group effect size for diCBT (Cohen's d) for primary outcome measure
Herbst et al. (2016) OCD eN = 34 eage = 36 Y-BOCS SR, fOCI-R, BDI-II, BSI, SCL-90R, PGI-I e8/e14 eiCBT vs. waiting list control ed = 0.82
Andersson et al. (2015) OCD N = 101 age = 35 Y-BOCS, OCI-R 10/10 prescribed individually fiCBT vs. online supportive therapy fd = 1.55
Bergman Nordgren et al. (2013)
Anxiety (eany anxiety disorder except PTSD or OCD)
N = 54 age = 39 CORE-OM, gMADRS-S, BAI, QOLI 10/6-10 prescribed individually giCBT vs. control group with access to online discussion forum gd = 1.15
Andersson et al. (2012b)
Depression N = 88 age = 39 BDI, hMADRS-S, BAI, QOLI 8/h7 hiCBT vs. e-mail vs. waiting list control hd = 1.46
GAD N = 89 iage = 39 iPSWQ STAI-S, STAI-T, BAI, BDI, QOLI 8/i8 iiCBT vs. waiting list control id = 1.08
SAD N = 204 age = 38 L-SAS-SR, jSPS, SIAS, SPSQ 9/j9 jiCBT vs. control group with access to online discussion forum d = 0.97
a

OCD = Obsessive compulsive disorder, GAD = General Anxiety Disorder, SAD = Social Anxiety Disorder, PTSD = Post-Traumatic Stress Disorder.

b

Y-BOCS SR = Yale-Brown Obsessive Compulsive Scale Self Report, OCI-R = Obsessive-Compulsive Scale Short version, BDI-II = Beck Depression Inventory II, BSI = Brief Symptom Inventory, SCL-90R = Symptom Checklist-90 Revised, PGI-I = Patient Global Impression of Improvement, CORE-OM The Clinical Outcomes in Routine Evaluation Outcome Measure, MADRS-S = The Montgomery–Åsberg Depression Rating Scale Self Report, BAI = Beck Anxiety Inventory, QOLI = Quality of Life Inventory, PSWQ = Penn State Worry Questionnaire, STAI-S = State-Trait Anxiety Inventory (State), STAI-T = State-Trait Anxiety Inventory (Trait), L-SAS-SR = Liebowitz Social Anxiety Scale self-report version, SPS = Social Phobia Scale, SIAS = Social Interaction Anxiety scale, SPSQ = Social Phobia Screening Questionnaire.

c

RCT = Randomized Controlled Trial.

d

iCBT = internet-based cognitive behavior therapy treatment group.

e

Information from the original article (Herbst et al., 2014).

f

Information from the original article (E. Andersson et al., 2012).

g

Information from the original article (Carlbring et al., 2011).

h

Information from the original article (Vernmark et al., 2010).

i

Information from the original article (Paxling et al., 2011).

j

Information from the original article (Andersson et al., 2012a).

While all the alliance analyses were reported in secondary analysis articles, the original articles (Andersson et al., 2012, Andersson et al., 2012a, Carlbring et al., 2011, Herbst et al., 2014, Paxling et al., 2011, Vernmark et al., 2010) were studied in order to gather the study details and procedure. If the data were available in the alliance articles, this information was preferable. Information retrieved from the original analysis articles is marked in the Table 1, Table 2, with references given in footnotes. Any discrepancies were discussed among the authors (SP and KJ) until consensus was reached.

Table 2.

Therapeutic alliance-related study characteristics and alliance association with outcome.

Authors, astudied disorder Mean and percent of completed modules/total
Mean and percent clients completed/started biCBT
Sample characteristics Therapist education, time (per client) Mode and frequency of contact Other contact Time, measure & rating of alliance Alliance effect on outcome
Herbst et al. (2016)
OCD
cNot reported
cN = 30/34, 88%
cPartly clinician-referred, amount of self-referred not reported, comorbidities not reported, severe depression excluded cExperienced cognitive behavioral therapists, time used not reported cWeekly e-mail support, contact twice a week cTelephone pre-screening and diagnostic interview, face-to-face diagnosis confirmation, telephone contact in case of significant worsening Endpoint
WAI (12 items) = 4.08
Significant correlation r = 0.33 with change score p = 0.04 (for N = 34)
Andersson et al. (2015)
OCD
d7.28/10, 73%
dN = 44/50, 88%
dPartly clinician-referred, 88% self-referred, comorbidity included, extreme or minimal OCD excluded Final year psychology students d129 min E-mail 2–3 times a week dTelephone initial and diagnostic interview, SMS when not logged in for a week, telephone call if still not logged in, automatic SMS on each e-mail Week 3
WAI (12 items) = 65,71
High alliance score (> 69) predicted less symptoms, larger change scores p < 0.05(for N = 50)
Bergman Nordgren et al. (2013)
Anxiety
(eany anxiety disorder except PTSD or OCD)
e7.96/8.86, 90%
eN = 59%
eSelf-referred comorbidities allowed Final year psychology students, instructed 15 min per week Weekly e-mail support eFace-to-face diagnostic interview, all other contact via e-mail Start, week 3 and endpoint WAI-S (12 items), minor modification = 6 Significant correlation r = − 0.47 with residual gain scores p = 0.019 (for N = 27)
Andersson et al. (2012b)
Depression f6/7, 86%
fN = 17/29, 59%
fSelf-referred, comorbidity included (38%), extreme or minimal depression excluded fFinal year psychology students 53 min fWeekly e-mail support fFace-to-face diagnostic interview, e-mail reminders, telephone call if not reached for two weeks (n = 3) Week 3
WAI (36 items), minor modification = 5,25
Residual gain score correlation r = 0.20 ns. (for N = 29)
GAD g4.8/8, 60%
g10.5%
gSelf-referred, comorbidity included (co morbid depression 22,5%) minimal GAD excluded gFinal year psychology students 97 min Weekly e-mail support gTelephone diagnostic interview, e-mail reminders Week 3
WAI (12 items) = 5,63
Residual gain score correlation r = 0.13 ns. (for N = 44)
SAD h6.8/9, 86%
hN = 56/102, 55%
hSelf-referred, comorbidity included but unknown hFinal year psychology students and licensed psychologists, on average 15 min per week Weekly e-mail support hTelephone diagnostic interview, SMS reminders Week 4
WAI (12 items) = 5,45
Residual gain score correlation r = 0.10 ns. (for N = 102)
a

OCD = Obsessive compulsive disorder, GAD = General Anxiety Disorder, SAD = Social Anxiety Disorder, PTSD = Post-Traumatic Stress Disorder.

b

iCBT = internet-based cognitive behavior therapy treatment.

c

Information from the original article (Herbst et al., 2014).

d

Information from the original article (E. Andersson et al., 2012).

e

Information from the original article (Carlbring et al., 2011).

f

Information from the original article (Vernmark et al., 2010).

g

Information from the original article (Paxling et al., 2011).

h

Information from the original article (Andersson et al., 2012a).

3. Results

3.1. Study characteristics

Basic study characteristics are summarized in Table 1. Treatments were similar in length, lasting from 8 to 10 weeks. Randomized control trial design (RCT) was used in all studies. The iCBT treatment was compared with either a control treatment or a waiting-list control group. Mean ages of participants (35 to 39 years) were comparable, and sample sizes of trials ranged from 34 to 204 participants. All studies reported alliance measures for the intervention group numbering 27 to 102, with the exception of Herbst et al. (2016), who measured alliance both in the intervention group of 18 and the control group of 16 (Table 2). All iCBT treatments were efficacious, with large effect sizes ranging from 0.82 to 1.55 in Cohen's d-value on the main outcome measures.

3.2. Therapeutic alliance-related study characteristics

Factors possibly affecting therapeutic alliance are presented in Table 2. Adherence was reported for all trials. All original studies except one (Herbst et al., 2014) reported mean of completed modules for the iCBT treatment group, and percentages were calculated based on that, varying from 60% to 90%. Numbers or percentages of completers of iCBT were also reported in original trials, except for one (Carlbring et al., 2011), that only reported completer percentage of both treatment and control groups. Completer percentage in treatment groups varied from 10,5% to 88%. Study populations were mostly self-referred, subjects being recruited by use of websites and the national media, with only two studies including some proportion of clinician-referred clients (Andersson et al., 2015, Herbst et al., 2016). All the studies excluded serious comorbidities like suicidality and history of psychosis, but some other comorbidities were allowed and are listed here if reported. Drop-out rates were low but, due to varying criteria, not directly comparable.

Majority of internet therapists were psychology students, but two studies employed either some clinical psychologists with experience in iCBT (Andersson et al., 2012b), or well-experienced cognitive-behavioral therapists (Herbst et al., 2016). In all studies, the main modality of contact with the internet therapist was e-mail. In four studies, e-mail was sent once a week and in the remaining two (both including a partly clinical population) (Andersson et al., 2015, Herbst et al., 2016), twice or three times a week. Both studies found a significant positive association between alliance and outcome. The time that internet therapists devoted to each client varied, ranging from 53 to 150 min but in one study went unreported (Herbst et al., 2016). In two other trials (Andersson et al., 2012b, Bergman Nordgren et al., 2013), the amount of support was not reported, but instead, therapists were instructed to spend 15 min per session supporting a client.

Each trial employed a diagnostic interview either by telephone or face-to-face, or both (Table 2). In some trials, SMS reminders and telephone calls were the choice if log-in was delayed despite e-mail and SMS reminders (Andersson et al., 2015, Andersson et al., 2012b). One study reported using telephone calls in cases in which significant decline in the client's condition emerged (Herbst et al., 2016).

3.3. Therapeutic alliance and it's association to treatment outcome and adherence

For therapeutic alliance measurement, most studies used the 12-item short version of the Working Alliance Inventory (Bordin, 1979). Only one trial (Andersson et al., 2012b) employed the full 36-item version of the WAI, while two others made some minor modifications to adjust the WAI for iCBT (Andersson et al., 2012b, Bergman Nordgren et al., 2013). The short version has been as valid as the 36-item version (Busseri and Tyler, 2003). Only one study (Herbst et al., 2016) reported the subscales of task, goal and bond of the WAI, so analysis of the subscales was not included in this review. Other studies only reported the mean alliance rating, reporting high intercorrelations of the WAI subscales. All studies used the mean alliance rating of WAI for the analysis of alliance association with outcome.

Three of the trials found that therapeutic alliance was associated with the primary outcome measure (Andersson et al., 2015, Bergman Nordgren et al., 2013, Herbst et al., 2016). Alliance ratings at mid-treatment predicted the outcome in two trials. A stronger mid-treatment WAI score correlated with symptom score changes (Bergman Nordgren et al., 2013) or predicted both score change and lower symptom scores at the end-point (Andersson et al., 2015). In one trial where alliance correlated with the outcome, the measurement of alliance was made only at the end of treatment (Herbst et al., 2016). In three other trials, all reported in the same alliance article (Andersson et al., 2012b), no significant relationships between the therapeutic alliance and treatment outcomes were found.

Adherence to iCBT treatment was reported in the original articles in terms of either proportion of completed modules or proportion of treatment completers or both. All studies except one (Herbst et al., 2014) reported mean of completed modules, with the percentages varying from 60% to 90%. Proportion of iCBT completers were reported as well, with the only exception of the study by Carlbring et al. (2011), who only reported pooled completer percentage of both treatment and control groups. Proportion of completers varied from 10,5% to 88%. No assessment of therapeutic alliance association with adherence was made.

4. Discussion

This review summarized current findings on client-therapist therapeutic alliance association with treatment outcome in therapist-supported iCBT programs for individual adult clients suffering from depressive or anxiety disorders. Research on this subject appears to be scarce: only four published articles reporting six different iCBT trials fulfilled the criteria of this review. This is in line with findings of earlier reviews on this subject, although they also included some other (than iCBT) internet interventions (Berger, 2015, Sucala et al., 2012). The majority of the studies analyzed here was conducted by the same Swedish research group and presumably deployed a similar type of program. Although therapeutic alliance is expected to enhance engagement and adherence to treatment (Christensen et al., 2002, Richards and Richardson, 2012, Simpson et al., 2011), none of the studies explored the relationship between the alliance and adherence.

What is worth mentioning here is that alliance measured in the studies reviewed here was defined specifically as the client-therapist relationship. The general view is that a therapeutic alliance can also occur between the client and the iCBT program itself and that this client-program alliance may be confused with that between client and therapist (Cavanagh, 2010, Cavanagh and Millings, 2013). However, the Working Alliance Inventory (WAI) – an alliance measure used in all reports studied – evaluates specifically the client-therapist relationship, while the client-program alliance remained beyond the focus of the current review. It has been suggested that the WAI subscales of agreement on goals and tasks would be important in internet interventions, while the affective bond between client and therapist would not (Berger, 2015). Since associations with the WAI subscales and outcome were not reported in the reviewed studies, this question could not be assessed here. Furthermore, the studies reported high intercorrelations with the WAI subscales, so this suggestion gained no support here.

Even despite the absence of any audio- or visual contact with the therapist and limited overall contact time, the iCBT patients reported high levels of therapeutic alliance, comparable to that in face-to-face therapies (Berger, 2015). These results are in line with those of the existing reviews (Berger, 2015, Sucala et al., 2012). This raises a major theoretical question as to the nature of the therapeutic relationship as an agent of change and engagement during the emerging era of digitalization, artificial intellect, and robotization (Cavanagh and Millings, 2013). How likely is it that in future, when a client modifies a therapeutic alliance concurrently with both a human and a machine, the share and role of the machine will grow? This question should be answered with novel, pertinent methodology in research yet to come.

4.1. Therapeutic alliance, and associations with treatment outcome and adherence

Therapeutic alliance was high in all the six studies, as expected based on the existing literature (Berger, 2015, Sucala et al., 2012). This finding indicates once again that therapeutic alliance can also be built and maintained in iCBTs. This information is important for the concept of alliance itself, since alliance is considered a vital ingredient in all successful psychotherapies (Ackerman et al., 2001).

In the three most recent trials, therapeutic alliance showed an association with treatment outcome (Andersson et al., 2015, Bergman Nordgren et al., 2013, Herbst et al., 2016). Alliance strength seems to predict outcome, but this observation is based only on two of these studies, since in the third, alliance was measured only at the end-point of the trial, not in mid-treatment (Herbst et al., 2016). In this case, therapeutic alliance cannot, with certainty, be considered a predictor, since any post-treatment alliance may be a consequence of treatment (Andersson et al., 2012b, Feeley et al., 1999). In the three earlier trials (Andersson et al., 2012b), positive correlations failed to reach statistical significance. These findings are concordant with those of earlier reviews on the subject (Berger, 2015, Sucala et al., 2012) reporting positive, but not always statistically significant correlations.

Adherence as proportion of completers in this review varied from 10,5% to 88%. This is roughly comparable to previous studies on depressive and anxiety disorders reporting attrition ranging from 1–50% (Christensen et al., 2009). Strikingly, although it is expected that alliance building may support engagement to treatment (Hilvert-Bruce et al., 2012, Richards and Richardson, 2012) and clinician contact during iCBT is associated with an increased adherence (Hilvert-Bruce et al., 2012), studies concerning association of therapeutic alliance and adherence are still missing.

4.2. Factors that may affect therapeutic alliance in iCBT

4.2.1. Methods

All the three trials that found no significant alliance-outcome association (Andersson et al., 2012b) (as opposed to those who did) (Andersson et al., 2015, Bergman Nordgren et al., 2013, Herbst et al., 2016) were conducted by the same researchers and used presumably similar methods. This could mean the method had some effect on the findings. As possible explanations for the lack of this association the authors discussed lack of statistical power or too small a range of alliance ratings, as well as a possible overlap between the WAI and symptom scores. Interestingly, the same research group did find significant alliance-outcome associations in their later studies. The reason for this difference in the results is unclear, but it could result from improved methodology. For instance, in the later study, the authors used change scores (Andersson et al., 2015) (vs residual gain score correlation earlier) and as a main outcome measure used the CORE-OM (Bergman Nordgren et al., 2013) - a measure of psychological distress, popular in studies of face-to-face psychotherapies.

4.2.2. Trial populations

All trials (Andersson et al., 2015, Andersson et al., 2012b, Bergman Nordgren et al., 2013, Herbst et al., 2016) recruited clients via websites and the national media. In two of these (Andersson et al., 2015, Herbst et al., 2016), the population was at least in part clinician-referred. In the earlier one (Andersson et al., 2015), 88% of the sample were self-referred. The later (Herbst et al., 2016), did not report the proportion of clinician-referred clients. These studies both found a significant positive association of alliance with outcome. Clinician-referred clients may be more likely to benefit from therapeutic alliance, since they already may have had more difficulties with engagement with treatment (Richards and Richardson, 2012) than did the self-referred volunteer clients, who tend to be a priori motivated to take the iCBT as their treatment of choice. This may also be true for the study by Bergman and collaborators (Bergman Nordgren et al., 2013) who included some clinician-referred clients and found high pre-treatment alliance scores. Indeed, self-referral is mentioned as an explanation for high alliance ratings in the very beginning of treatment (Berger, 2015).

One suggestion is that clients with comorbidities benefit from additional support (see Blom et al., 2016). For such clients, therapeutic alliance may, hence, improve outcome via better engagement with treatment (Richards and Richardson, 2012). In this review, the only study that freely allowed comorbidities (Bergman Nordgren et al., 2013) demonstrated statistically significant alliance-outcome associations.

4.2.3. Disorders

The associations of alliance with outcome emerged in both trials of OCD (Andersson et al., 2015, Herbst et al., 2016) and in the only trial on several different anxiety disorders (Bergman Nordgren et al., 2013). Notably, both OCD studies included challenging elements (exposure and response prevention), for which sufficient support for the clients is essential (Andersson et al., 2012b, Herbst et al., 2014). This additional support given by the therapists in these OCD trials could foster therapeutic alliance.

No alliance-outcome associations emerged in trials on depression, general anxiety disorder, or social anxiety disorder (Andersson et al., 2012b). Given the small number of studies and the fact that they were all carried out by the same research group (and presumably with similar programs and methodology), to reach firm conclusions on the disorder-specific effects of alliance in iCBT is difficult.

4.2.4. Therapist-client contact frequency and time spent by therapist

The time used by a therapist per client seemed to range in the reviewed studies, as expected, from 1.5 to 2.5 h during the whole program (Cavanagh and Millings, 2013). One trial reported less than 1 h of support per client (Andersson et al., 2012b) during all seven modules of the program, with no alliance-outcome association found, but two other studies with more than 2 h of therapist time per client, did find such an association (Andersson et al., 2015). Moreover, the latter study, rather than the more typical once a week, reported contact with the client 2 to 3 times a week.

Another study with a positive alliance-outcome relationship included 14 modules and contact 2–3 times per week, meaning surely a prolonged overall contact time. However, the therapist time used was not reported (Herbst et al., 2016).

A recent review concerning iCBT has concluded that therapist time used may not be critical in terms of treatment outcome (Baumeister et al., 2014). According to our findings based on more recent trials (Andersson et al., 2015, Bergman Nordgren et al., 2013), a prolonged client-therapist iCBT contact time may, however, strengthen the alliance and thereby alliance-outcome association. Nevertheless, also our data are not sufficient for making solid conclusions on the contact time vs outcome effects.

4.2.5. Mode of communication during treatment

Text-based communication served as the main contact mode in all trials. Of the three studies that found no positive alliance-outcome association, two studies included face-to-face screening diagnostic interviews with the study personnel (Bergman Nordgren et al., 2013, Herbst et al., 2016). The third one used SMS reminders in addition to normal e-mail communication and, if needed, telephone calls (Andersson et al., 2015). Of other three trials with no such associations (Andersson et al., 2012b), one employed a face-to-face diagnostic interview, and two other trials performed diagnostic interviews by telephone. One of the latter two trials also utilized additional SMSes. The dose and effects of different communication modes sufficient for optimal level of alliance and thereby, plausibly for optimal outcomes thus remain still obscure and should be a subject of future research.

4.2.6. Internet therapists

One recent review found no therapist qualification-dependent differences in terms of treatment outcome (Baumeister et al., 2014). In one large meta-analysis on the iCBT for depression, a higher drop-out rate emerged, however, if a therapist was an administration employee rather than a clinical professional (Richards and Richardson, 2012). No studies have, however, reported data on the role of internet therapist qualification for therapeutic alliance as a possible mediator of outcome.

In the studies reviewed here, all internet therapists were at least Masters level psychology students or even were qualified cognitive-behavioral therapists. This prevents any between-trial comparison of the effects of therapist qualification on alliance.

Furthermore, conventional therapist training tends to focus on face-to-face therapy and may not be the optimal in the iCBT setting. Studies explored here reported therapist education but failed to report other factors like motivation or experience in working online. The issue of association between internet therapist's education and therapeutic alliance has as yet been neglected, and should be explored in future research.

4.2.7. Qualities of the iCBT programs

The iCBT programs differ in terms of their alliance-related features, depending on treatment focus (Barazzone et al., 2012). Some programs may be designed more to convey a traditional therapeutic model emphasizing engagement and some others rather as an interactive educational tool. Barrazzone and collaborators have shown that iCBT programs themselves contain some relational features (Barazzone et al., 2012), since even the self-help text can convey a sense of understanding, warmth, and empathy (Andersson et al., 2012b, Richardson et al., 2010). All the six studies explored here reported only the client-therapist alliance, not the client-program alliance. What could be intriguing, however, is whether the client-program alliance has an add-on impact on the client-therapist alliance.

4.3. Issues in research on therapeutic alliance in iCBT programs and future directions

Research on alliance with internet therapist in iCBT faces several future challenges. First, some alliance-related issues (such as therapist time required) were not reported. Second, all studies were performed with the RCT design – the gold standard of intervention studies which cannot, however, be directly extrapolated into real world practice and policy. Third, the data cannot be directly translated into usual clinical practice also due to the populations studied – solely or mostly self-referred clients. Fourth, while client assessments of alliance tend to be explored sufficiently, assessments by the internet therapist or by independent raters are usually missing. Fifth, the client experience of the support – probably a key issue for alliance modification – is reported in only one of these studies (Herbst et al., 2016). Sixth, despite the ever-growing interest in and body of reports on therapeutic alliance in the iCBT, only a few studies within the inclusion criteria of the current review have investigated alliance-outcome associations. Seventh, several pre-, on-, and post-therapy measuring points could provide theoretically important information on the prospective pipeline of alliance development, but only one of the articles (Bergman Nordgren et al., 2013) provided such data. Measuring alliance before treatment, though methodologically challenging (Berger, 2015), may prove useful, since high alliance pre-treatment ratings may reflect positive expectations towards treatment rather than the alliance itself. On the other hand, alliance measured only at the end-point may be a result of treatment rather than a predictor or mediator of outcome (Andersson et al., 2012b, Cavanagh, 2010). This means that a more reliable outcome predictor or mediator may be the mid-treatment outcome measure. Eighth, an even more challenging issue may be the measuring and detailed description of the characteristics of the iCBT therapist her- or himself (vs extensively studied therapist effects in face-to-face therapy) (Lambert, 2013, Wampold and Imel, 2015), and the iCBT program as such, which leaves a range of probable predictors of the alliance (and possibly of outcome) beyond scientific scrutiny. Finally, effects of the optimal mode and amount of iCBT therapist-provided support are obscure. The need for and importance of the support may vary by client, depending on such factors as his or her passive vs active approach towards treatment (Halmetoja et al., 2014) or other still unknown characteristics affecting willingness to interact with the therapist (Berger, 2015). Future research should elucidate client characteristics critical for development of therapeutic alliance and the support needed to achieve optimal clinical results (Berger et al., 2011, Oromendia et al., 2016).

5. Limitations

This review was based on searches in five databases (PubMed, PsycINFO, SCOPUS, Cochrane Library and CINAHL) and included only English-language studies on internet therapy programs for common mental disorders. Furthermore, only individual interventions with individual support were chosen for examination. Moreover, we required that guidance, support, or alliance were mentioned in the abstract. Thus it is possible that some relevant papers were missed. Since studies on different mental disorders were included, with only one study for each (with the exception of the OCD with two reports) it was not possible to draw disorder-specific conclusions. Due to the lack of relevant reports, it was impossible to elucidate the plausible effect of alliance on the treatment adherence in the iCBT – a potentially valuable direction for future research.

6. Conclusions

Therapeutic alliance in iCBT is high and may be even stronger than in face-to-face therapy. However, relationship between therapeutic alliance in and outcome of iCBT is still understudied. Nevertheless, it appears that, apart from results in one article, therapeutic alliance is directly associated with treatment outcome. Studies on effects of alliance on treatment adherence are, thus far, unavailable, although this piece of knowledge could be important to define the optimal human support and to fully utilize the potential of iCBT. The structured internet therapy programs with limited, text-based human support may be a plentiful environment for exploration of various elements of therapeutic alliance and can enhance our understanding of the nature and concept of the alliance itself.

The significance and role of therapeutic alliance in guided internet therapy programs for mental disorders is worthy of thorough research for reasons both theoretical and practical.

Disclosure of interest

Concerning the article manuscript Therapeutic alliance in guided internet therapy programs for depression and anxiety disorders – a systematic review the authors Satu Pihlaja, Jan-Henry Stenberg, Kaisla Joutsenniemi, Heidi Mehik, Ville Ritola and Grigori Joffe thereby declare, that there are no conflicts of interest.

This study was conducted in accordance with the guidelines by the Finnish Advisory Board on Research Integrity on the responsible conduct of research and procedures.

Acknowledgements

We would like to thank Doctor Carolyn Brimley Norris for the English language check of this article.

This work was partly supported by Government of Finland (research grant TYH2015218).

Contributor Information

Satu Pihlaja, Email: satu.pihlaja@gmail.com.

Jan-Henry Stenberg, Email: jan-henry.stenberg@hus.fi.

Kaisla Joutsenniemi, Email: kaisla.joutsenniemi@hus.fi.

Heidi Mehik, Email: heidi.mehik@hus.fi.

Ville Ritola, Email: ville.ritola@hus.fi.

Grigori Joffe, Email: grigori.joffe@hus.fi.

References

  1. Aboujaoude E., Salame W., Naim L. Telemental health: a status update. World Psychiatry. 2015;14:223–230. doi: 10.1002/wps.20218. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ackerman S.J., Benjamin L.S., Beutler L.E., Gelso C.J., Goldfried M.R., Hill C.…Rainer J. Empirically supported therapy relationships: conclusions and recommendations of the Division 29 Task Force. Psychother. Theory Res. Pract. Train. 2001;38:495–497. [Google Scholar]
  3. Andersson G., Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn. Behav. Ther. 2009;38:196–205. doi: 10.1080/16506070903318960. [DOI] [PubMed] [Google Scholar]
  4. Andersson E., Enander J., Andrén P., Hedman E., Ljótsson B., Hursti T.…Rück C. Internet-based cognitive behaviour therapy for obsessive-compulsive disorder: a randomized controlled trial. Psychol. Med. 2012;42:2193–2203. doi: 10.1017/S0033291712000244. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Andersson G., Carlbring P., Furmark T., on behalf of the S. O. F. I. E. Research Group Therapist experience and knowledge acquisition in internet-delivered CBT for social anxiety disorder: a randomized controlled trial. PLoS One. 2012;7 doi: 10.1371/journal.pone.0037411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Andersson G., Paxling B., Wiwe M., Vernmark K., Felix C.B., Lundborg L.…Cuijpers P. Therapeutic alliance in guided internet-delivered cognitive behavioural treatment of depression, generalized anxiety disorder and social anxiety disorder. Behav. Res. Ther. 2012;50:544–550. doi: 10.1016/j.brat.2012.05.003. [DOI] [PubMed] [Google Scholar]
  7. Andersson G., Carlbring P., Ljótsson B., Hedman E. Guided internet-based CBT for common mental disorders. J. Contemp. Psychother. 2013;43:223–233. [Google Scholar]
  8. Andersson G., Cuijpers P., Carlbring P., Riper H., Hedman E. Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. 2014;13:288–295. doi: 10.1002/wps.20151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Andersson E., Ljótsson B., Hedman E., Enander J., Kaldo V., Andersson G.…Rück C. Predictors and moderators of internet-based cognitive behavior therapy for obsessive-compulsive disorder: results from a randomized trial. J. Obsessive Compuls. Relat. Disord. 2015;4:1–7. [Google Scholar]
  10. Andrews G., Cuijpers P., Craske M.G., McEvoy P., Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010;13 doi: 10.1371/journal.pone.0013196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Barak A., Hen L., Boniel-Nissim M., Shapira N.A. A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. J. Technol. Hum. Serv. 2008;26:109–160. [Google Scholar]
  12. Barak A., Klein B., Proudfoot J.G. Defining internet-supported therapeutic interventions. Ann. Behav. Med. 2009;38:4–17. doi: 10.1007/s12160-009-9130-7. [DOI] [PubMed] [Google Scholar]
  13. Barazzone N., Cavanagh K., Richards D.A. Computerized cognitive behavioural therapy and the therapeutic alliance: a qualitative enquiry. Br. J. Clin. Psychol. 2012;51:396–417. doi: 10.1111/j.2044-8260.2012.02035.x. [DOI] [PubMed] [Google Scholar]
  14. Baumeister H., Reichler L., Munzinger M., Lin J. The impact of guidance in internet-based mental health interventions – a systematic review. Internet Interv. 2014;1:205–215. [Google Scholar]
  15. Baxter A.J., Vos T., Scott K.M., Ferrari A.J., Whiteford H.A. The global burden of anxiety disorders in 2010. Psychol. Med. 2014;44:2363–2374. doi: 10.1017/S0033291713003243. [DOI] [PubMed] [Google Scholar]
  16. Berger T. The therapeutic alliance in internet interventions: a narrative review and suggestions for future research. Psychother. Res. 2015;6:1–14. doi: 10.1080/10503307.2015.1119908. [DOI] [PubMed] [Google Scholar]
  17. Berger T., Caspar F., Richardson R., Kneubuehler B., Sutter D., Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. Behav. Res. Ther. 2011;49:158–169. doi: 10.1016/j.brat.2010.12.007. [DOI] [PubMed] [Google Scholar]
  18. Berger T., Boettcher J., Caspar F. Internet-based guided self-help for several anxiety disorders: a randomized controlled trial comparing a tailored with a standardized disorder-specific approach. Psychotherapy. 2014;51:207–219. doi: 10.1037/a0032527. [DOI] [PubMed] [Google Scholar]
  19. Bergman Nordgren L., Carlbring P., Linna E., Andersson G. Role of the working alliance on treatment outcome in tailored internet-based cognitive behavioural therapy for anxiety disorders: randomized controlled pilot trial. J. Med. Internet Res. 2013;18 doi: 10.2196/resprot.2292. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Blom K., Jernelöv S., Lindefors N., Kaldo V. Facilitating and hindering factors in internet-delivered treatment for insomnia and depression. Internet Interv. 2016;4:51–60. doi: 10.1016/j.invent.2016.03.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Bordin E.S. The generalizability of the psychoanalytic concept of the working alliance. Psychother. Theory Res. Pract. Train. 1979;16:252–260. (Retrieved from http://ovidsp.uk.ovid.com) [Google Scholar]
  22. Bordin E. Theory and research on the therapeutic working alliance: new directions. In: Horvath A.O., Greenberg L.S., editors. The Working Alliance: Theory, Research, and Practice. Wiley; New York: 1994. pp. 13–37. [Google Scholar]
  23. Busseri M.A., Tyler J.D. Interchangeablility of the working alliance inventory and working alliance inventory, short form. Psychol. Assess. 2003;15:193–197. doi: 10.1037/1040-3590.15.2.193. [DOI] [PubMed] [Google Scholar]
  24. Carlbring P., Maurin L., Törngren C., Linna E., Eriksson T., Sparthan E.…Marquez C. Individually-tailored, internet-based treatment for anxiety disorders: a randomized controlled trial. Behav. Res. Ther. 2011;49:18–24. doi: 10.1016/j.brat.2010.10.002. [DOI] [PubMed] [Google Scholar]
  25. Cavanagh K. Turn on, tune in and (don't) drop out: engagement, adherence, attrition, and alliance with Internet-based interventions. In: Bennett-Levy J., Richards D.A., Farrand P., Christensen H., Griffiths K.M., Williams C., editors. Oxford Guide to Low Intensity CBT Interventions. Oxford University Press; New York: 2010. pp. 227–233. [Google Scholar]
  26. Cavanagh K., Millings A. (Inter)personal computing: the role of the therapeutic relationship in e-mental health. J. Contemp. Psychother. 2013;43:197–206. [Google Scholar]
  27. Christensen H., Griffiths K.M., Korten A. Web-based cognitive behavior therapy: analysis of site usage and change in depression and anxiety scores. J. Med. Internet Res. 2002;4 doi: 10.2196/jmir.4.1.e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Christensen H., Griffiths K.M., Jorm A.F. Delivering interventions for depression by using the internet: randomised controlled trial. Br. Med. J. 2004;31:265. doi: 10.1136/bmj.37945.566632.EE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Christensen H., Griffiths K.M., Farrer L. Adherence in internet interventions for anxiety and depression: systematic review. J. Med. Internet Res. 2009;11 doi: 10.2196/jmir.1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Clarke J., Proudfoot J., Whitton A., Birch M.-R., Boyd M., Parker G.…Fogarty A. Therapeutic alliance with a fully automated mobile phone and web-based intervention: secondary analysis of a randomized controlled trial. J. Med. Internet Res. 2016;3 doi: 10.2196/mental.4656. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Cuijpers P., Marks I.M., van Straten A., Cavanagh K., Gega L., Andersson G. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn. Behav. Ther. 2009;38:66–82. doi: 10.1080/16506070802694776. [DOI] [PubMed] [Google Scholar]
  32. Feeley M., DeRubeis R.J., Gelfand L.A. The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression. J. Consult. Clin. Psychol. 1999;67:578–582. doi: 10.1037//0022-006x.67.4.578. [DOI] [PubMed] [Google Scholar]
  33. Hadjistavropoulos H.D., Pugh N.E., Hesser H., Andersson G. Therapeutic alliance in internet-delivered cognitive behaviour therapy for depression or generalized anxiety. Clin. Psychol. Psychother. 2016;24:451–461. doi: 10.1002/cpp.2014. [DOI] [PubMed] [Google Scholar]
  34. Halmetoja C.O., Malmquist A., Carlbring P., Andersson G. Experiences of internet-delivered cognitive behavior therapy for social anxiety disorder four years later: a qualitative study. Internet Interv. 2014;1:158–163. [Google Scholar]
  35. Hattema J.M. What is the genetic relationship between anxiety and depression? Am. J. Med. Genet. C: Semin. Med. Genet. 2008;15:140–147. doi: 10.1002/ajmg.c.30171. [DOI] [PubMed] [Google Scholar]
  36. Herbst N., Voderholzer U., Thiel N., Schaub R., Knaevelsrud C., Stracke S.…Külz A.K. No talking, just writing! Efficacy of an internet-based cognitive behavioral therapy with exposure and response prevention in obsessive compulsive disorder. Psychother. Psychosom. 2014;83:165–175. doi: 10.1159/000357570. [DOI] [PubMed] [Google Scholar]
  37. Herbst N., Franzen G., Voderholzer U., Thiel N., Knaevelsrud C., Hertenstein E.…Külz A. Working alliance in internet-based cognitive-behavioral therapy for obsessive-compulsive disorder. Psychother. Psychosom. 2016;85:117–118. doi: 10.1159/000441282. [DOI] [PubMed] [Google Scholar]
  38. Hilvert-Bruce Z., Rossouw P.J., Wong N., Sunderland M., Andrews G. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav. Res. Ther. 2012;50:463–468. doi: 10.1016/j.brat.2012.04.001. [DOI] [PubMed] [Google Scholar]
  39. Kaufman J., Charney D. Comorbidity of mood and anxiety disorders. Depress. Anxiety. 2000;12:69–76. doi: 10.1002/1520-6394(2000)12:1+<69::AID-DA9>3.0.CO;2-K. [DOI] [PubMed] [Google Scholar]
  40. Kiropoulos L.A., Klein B., Austin D.W., Gilson K., Pier C., Mitchell J.…Ciechomski L. Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT? J. Anxiety Disord. 2008;22:1273–1284. doi: 10.1016/j.janxdis.2008.01.008. [DOI] [PubMed] [Google Scholar]
  41. Knaevelsrud C., Maercker A. Does the quality of the working alliance predict treatment outcome in online psychotherapy for traumatized patients? J. Med. Internet Res. 2006;8 doi: 10.2196/jmir.8.4.e31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Kohn R., Saxena S., Levav I., Saraceno B. The treatment gap in mental health care. Bull. World Health Organ. 2004;82:858–866. [PMC free article] [PubMed] [Google Scholar]
  43. Lambert M.J. Outcome in psychotherapy: the past and important advance. Psychotherapy. 2013;50:42–51. doi: 10.1037/a0030682. [DOI] [PubMed] [Google Scholar]
  44. Levine J., Cole D.P., Chengappa K.N., Gershon S. Anxiety disorders and major depression, together or apart. Depress. Anxiety. 2001;14:94–104. doi: 10.1002/da.1051. [DOI] [PubMed] [Google Scholar]
  45. Leykin Y., Derubeis R.J., Gallop R., Amsterdam J.D., Shelton R.C., Hollon S.D. The relation of patients' treatment preferences to outcome in a randomized clinical trial. Behav. Ther. 2007;38:209–217. doi: 10.1016/j.beth.2006.08.002. [DOI] [PubMed] [Google Scholar]
  46. Lindner P., Olsson E.L., Johnsson A., Dahlin M., Andersson G., Carlbring P. The impact of telephone versus e-mail therapist guidance on treatment outcomes, therapeutic alliance and treatment engagement in internet-delivered CBT for depression: a randomised pilot trial. Internet Interv. 2014;(4):182–187. [Google Scholar]
  47. MacLeod M., Martinez R., Williams C. Cognitive behaviour therapy self-help: who does it help and what are its drawbacks? Behav. Cogn. Psychother. 2009;37:61–72. doi: 10.1017/S1352465808005031. [DOI] [PubMed] [Google Scholar]
  48. McHugh R.K., Whitton S.W., Peckham A.D., Welge J.A., Otto M.W. Patient preference for psychological vs. pharmacological treatment of psychiatric disorders: a meta-analytic review. J. Clin. Psychiatry. 2013;74:595–602. doi: 10.4088/JCP.12r07757. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Mechanic D. Barriers to help-seeking, detection, and adequate treatment for anxiety and mood disorders: implications for health care policy. J. Clin. Psychiatry. 2007;68:20–26. http://www.psychiatrist.com/jcp/article/Pages/2007/v68s02/v68s0203.aspx Retrieved from. [PubMed] [Google Scholar]
  50. Melville K.M., Casey L.M., Kavanagh D.J. Dropout from internet-based treatment for psychological disorders. Br. J. Clin. Psychol. 2010;49:455–471. doi: 10.1348/014466509X472138. [DOI] [PubMed] [Google Scholar]
  51. National Institute for Health and Care Excellence . NICE Clinical Guideline 159. Author; Manchester, NH: 2013. Social anxiety disorder: recognition, assessment and treatment.https://www.ncbi.nlm.nih.gov/pubmed/25577940 Retrieved from. [PubMed] [Google Scholar]
  52. Norcross J.C. 2nd ed. Oxford University Press; New York: 2011. Psychotherapy Relationships That Work. [Google Scholar]
  53. Oromendia P., Orrego J., Bonillo A., Molinuevo B. Internet-based self-help treatment for panic disorder: a randomized controlled trial comparing mandatory versus optional complementary psychological support. Cogn. Behav. Ther. 2016;45:270–286. doi: 10.1080/16506073.2016.1163615. [DOI] [PubMed] [Google Scholar]
  54. Paxling B., Almlöv J., Dahlin M., Carlbring P., Breitholtz E., Eriksson T., Andersson G. Guided internet-delivered cognitive behaviour therapy for generalized anxiety disorder: a randomized controlled trial. Cogn. Behav. Ther. 2011;40:159–173. doi: 10.1080/16506073.2011.576699. [DOI] [PubMed] [Google Scholar]
  55. Reynolds D.A.J., Jr., Stiles W.B., Grohol J.M. An investigation of session impact and alliance in internet based psychotherapy: preliminary results. Couns. Psychother. Res. 2006;6:164–168. [Google Scholar]
  56. Richards D., Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin. Psychol. Rev. 2012;32:329–347. doi: 10.1016/j.cpr.2012.02.004. [DOI] [PubMed] [Google Scholar]
  57. Richardson R., Richards D.A., Barkham M. Self-help books for people with depression: the role of the therapeutic relationship. Behav. Cogn. Psychother. 2010;38:67–81. doi: 10.1017/S1352465809990452. [DOI] [PubMed] [Google Scholar]
  58. Rosellini A.J., Brown T.A. The NEO five-factor inventory: latent structure and relationships with dimensions of anxiety and depressive disorders in a large clinical sample. Assessment. 2011;18:27–38. doi: 10.1177/1073191110382848. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Saddichha S., Al-Desouki M., Lamia A., Linden I.A., Krausz M. Online interventions for depression and anxiety - a systematic review. Health Psychol. Behav. Med. 2014;2:841–881. doi: 10.1080/21642850.2014.945934. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Simpson S.G., Reid C.L. Therapeutic alliance in videoconferencing psychotherapy: a review. Aust. J. Rural Health. 2014;22:280–299. doi: 10.1111/ajr.12149. [DOI] [PubMed] [Google Scholar]
  61. Simpson H.B., Maher M.J., Wang Y., Bao Y., Foa E.B., Franklin M. Patient adherence predicts outcome from cognitive behavioral therapy in obsessive-compulsive disorder. J. Consult. Clin. Psychol. 2011;79:247–252. doi: 10.1037/a0022659. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Sucala M., Schnur J.B., Constantino M.J., Miller S.J., Brackman E.H., Montgomery G.H. The therapeutic relationship in e-therapy for mental health: a systematic review. J. Med. Internet Res. 2012;14:e110–187. doi: 10.2196/jmir.2084. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. van Schaik D.J.G., Klijn A.F.J., van Hout H.P.J., van Marwijk H.W.J., Beekman A.T.F., de Haan M. Patients' preferences in the treatment of depressive disorder in primary care. Gen. Hosp. Psychiatry. 2004;26:184–189. doi: 10.1016/j.genhosppsych.2003.12.001. [DOI] [PubMed] [Google Scholar]
  64. Vernmark K., Lenndin J., Bjärehed J., Carlsson M., Karlsson J., Öberg J.…Eriksson T. Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behav. Res. Ther. 2010;48:368–376. doi: 10.1016/j.brat.2010.01.005. [DOI] [PubMed] [Google Scholar]
  65. Wampold B.E., Imel Z.E. 2nd ed. Routledge; New York: 2015. The Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work. [Google Scholar]
  66. Whiteford H.A., Ferrari A.J., Degenhardt L., Feigin V., Vos T. The global burden of mental, neurological and substance use disorders: an analysis from the global burden of disease study 2010. PLoS One. 2015;10 doi: 10.1371/journal.pone.0116820. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. World Health Organization . WHO Health statistics and health information systems; Geneva: March 2011. Global Burden of Disease. [Google Scholar]
  68. World Health Organization Feb 2017. http://www.who.int/mediacentre/factsheets/fs369/en/
  69. Young A.S., Klap R., Sherbourne C.D., Wells K.B. The quality of care for depressive and anxiety disorders in the United States. Arch. Gen. Psychiatry. 2001;58:55–61. doi: 10.1001/archpsyc.58.1.55. [DOI] [PubMed] [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES