Skip to main content
PMC home page
Journal of Medical Internet Research logoLink to Journal of Medical Internet Research
. 2017 Sep 15;19(9):e306. doi: 10.2196/jmir.6588

Blending Face-to-Face and Internet-Based Interventions for the Treatment of Mental Disorders in Adults: Systematic Review

Doris Erbe 1,#, Hans-Christoph Eichert 1, Heleen Riper 2, David Daniel Ebert 3,✉
Editor: Gunther Eysenbach
Reviewed by: Joshua Fogel, Rosalie van der Vaart, Ulf-Dietrich Reips
PMCID: PMC5622288  PMID: 28916506

Abstract

Background

Many studies have provided evidence for the effectiveness of Internet-based stand-alone interventions for mental disorders. A newer form of intervention combines the strengths of face-to-face (f2f) and Internet approaches (blended interventions).

Objective

The aim of this review was to provide an overview of (1) the different formats of blended treatments for adults, (2) the stage of treatment in which these are applied, (3) their objective in combining face-to-face and Internet-based approaches, and (4) their effectiveness.

Methods

Studies on blended concepts were identified through systematic searches in the MEDLINE, PsycINFO, Cochrane, and PubMed databases. Keywords included terms indicating face-to-face interventions (“inpatient,” “outpatient,” “face-to-face,” or “residential treatment”), which were combined with terms indicating Internet treatment (“internet,” “online,” or “web”) and terms indicating mental disorders (“mental health,” “depression,” “anxiety,” or “substance abuse”). We focused on three of the most common mental disorders (depression, anxiety, and substance abuse).

Results

We identified 64 publications describing 44 studies, 27 of which were randomized controlled trials (RCTs). Results suggest that, compared with stand-alone face-to-face therapy, blended therapy may save clinician time, lead to lower dropout rates and greater abstinence rates of patients with substance abuse, or help maintain initially achieved changes within psychotherapy in the long-term effects of inpatient therapy. However, there is a lack of comparative outcome studies investigating the superiority of the outcomes of blended treatments in comparison with classic face-to-face or Internet-based treatments, as well as of studies identifying the optimal ratio of face-to-face and Internet sessions.

Conclusions

Several studies have shown that, for common mental health disorders, blended interventions are feasible and can be more effective compared with no treatment controls. However, more RCTs on effectiveness and cost-effectiveness of blended treatments, especially compared with nonblended treatments are necessary.

Keywords: mental health, Internet, psychotherapy, blended treatment

Introduction

Background

Empirical evidence suggests that Internet-based psychological interventions can be used to effectively treat adults, adolescents, and children for various mental disorders such as depression, anxiety, or problematic substance use [1-6]. Such interventions have several advantages over conventional face-to-face (f2f) interventions. For example, Internet-based interventions can be administered over long distances, may save therapists’ time, allow both patients and clinicians to work at their own pace, save traveling time, and reduce the stigma of having a mental disorder or going to a psychologist or therapist [7,8].

On the other hand, Internet-based interventions may also have disadvantages when compared with face-to-face therapies. For instance, Internet interventions may require certain abilities such as computer and Internet skills, reading and writing skills, and, in comparison with traditional therapy settings, more self-reflection and eloquence when talking about one’s thoughts and feelings. Furthermore, it has been argued that this type of intervention may make it difficult for therapists to adequately react to crisis situations such as suicidality because nonverbal cues are missing as additional information when assessing whether dissociation of suicidal thoughts is possible [9]. Also, negative effects such as frustration due to failure or time pressure might be associated with the Internet treatment format [10-13]. These disadvantages often lead to stand-alone Internet treatments being regarded as low-threshold interventions for milder cases of mental disorders, whereas face-to-face therapy and pharmacotherapy are often regarded as options of choice for more severe symptoms [14]. In addition to guided or unguided Internet treatment as a stand-alone intervention, a newer treatment approach combines face-to-face sessions with Internet-based sessions into one integrated treatment. This approach is usually called blended treatment and aims at retaining the positive aspects associated with both forms of therapy while mitigating the disadvantages [15-17]. There are different potential advantages of blending Internet and face-to-face treatments. For example, viewed from a cost-effectiveness perspective, blended treatments could possibly diminish the number of face-to-face contacts and thereby decrease the overall costs of treatment. Blended treatments could also increase the effectiveness by, for example, increasing frequency from one to two sessions per week [18] through adding Internet sessions to face-to-face interventions, thereby increasing intensity and success without additional costs. Furthermore, adding Internet interventions might improve transfer to everyday life as Internet or mobile elements could be used to support behavior change during face-to-face sessions and thereby increase effectiveness of face-to-face psychotherapy. Blended format could potentially also reach individuals for whom either purely delivered Internet-based or pure face-to-face approaches are not a suitable treatment option and thereby increase the utilization of effective treatments.

As both face-to-face and Internet-based psychotherapy have advantages and disadvantages, combining the two approaches in a blended treatment might combine the best of two worlds.

Definition of Blended Interventions

As a clear definition of blended interventions is still missing [17], we define blended interventions in this study as treatment programs that use elements of both face-to-face and Internet-based interventions, including both the integrated and the sequential use of both treatment formats. Nonblended interventions comprise face-to-face treatments only or stand-alone Internet treatments. Within blended treatments, face-to-face contacts may be added to Internet interventions or, vice versa, the Internet-based part may be arranged as an adjunct to existing face-to-face programs. Internet interventions may also be arranged as an aftercare or maintenance element after acute phase treatment or as an early step within a stepped care program. Stepped care refers to a treatment program where interventions start with the least intensive and least costly treatment that is likely to work, progressing step-by-step with more intensive interventions for those patients insufficiently helped by the first or previous intervention. Blended stepped care treatments involve Internet-based treatments as one step within the sequence. See Textbox 1 for a subject index of types of blended interventions.

Subject index.

Nonblended interventions: Face-to-face (f2f) treatments or stand-alone Internet-treatments only

Blended interventions: Treatment programs that use elements of both face-to-face and Internet-based interventions, including sequential use of both forms of treatment

  • Integrated blended interventions: Blended treatments where the Internet-based intervention part is arranged as an adjunct to face-to-face programs or vice versa so that face-to-face and Internet-based elements are provided within the same period. In integrated blended interventions, the focus can be either on the face-to-face treatment or on the Internet-based intervention.

  • Sequential blended interventions: Blended treatments where the Internet-based intervention part is arranged before or after the face-to-face treatment such as within stepped care approaches or aftercare interventions that directly follow the face-to-face intervention.

Objective

This systematic review supplies an overview of research into blended interventions for mental health. Specifically, we focus on the following questions: (1) Which blended intervention concepts have been proposed in the researched literature in the treatment of common mental disorders (anxiety disorders, depression, and substance abuse)? (2) In which stage of treatment (such as first step, acute phase treatment, and maintenance phase) do the Internet interventions take place? (3) Which types of problem and target group do the blended interventions focus on? (4) What is the objective in combining face-to-face and Internet-based approaches? and (5) What evidence is there for the effectiveness of blended interventions?

Methods

Search Strategy

Studies of potential relevance were identified using a systematic search in the MEDLINE, PsycINFO, Cochrane, and PubMed databases. All studies up to December 2015 were included. Searches were performed using keywords indicating face-to-face interventions (“inpatient,” “outpatient,” “face to face,” or “residential treatment”), which were combined with terms indicating Internet treatment (“internet,” “online,” or “web”) and terms indicating mental disorders (“mental health,” “depression,” “anxiety,” or “substance abuse”). We focused on three of the most common mental disorders: depression, anxiety, and substance abuse. The bibliographies of the identified studies revealed additional sources.

Study Selection

Studies were included if they met the following inclusion criteria: (1) the study was on an intervention that was based on both an Internet and a face-to-face treatment element that was either integrated or delivered sequentially, (2) the study involved treatment for adults with depression, anxiety, or substance abuse, and (3) the study was published in English or German. Studies with mere self-help interventions were excluded.

After the initial database search and removal of duplicates, the title and abstract of the remaining studies were rated for the inclusion criteria independently by the first and second author. Interrater reliability was good (kappa, κ=.825, P<.001). The first and second authors then used the inclusion criteria to independently review all full-text publications judged as relevant in the title and abstract screening. Consensus through discussion was sought in cases of disagreement. If no agreement could be reached through discussion, the last author made a decisive judgment. This was the case for 2 studies.

In November 2016, we started a second search in the database of PubMed, searching for studies citing the studies we had found in the initial search. The extraction strategies as of the initial search were used for those studies, including the independent rating of studies by the first and second authors in the first and second steps.

Quality Assessment

New quality assessment criteria were created since there are no current guidelines for assessing the quality of blended intervention studies. The quality of each study was rated on five aspects: study design, randomization of study conditions, report of statistics, sample size (studies powered to detect effect sizes of a minimal important difference between blended and nonblended interventions of a priori defined Cohen d of 0.35 were classified as high quality), and existence of a nonblended active control group. Each aspect was rated on a 3-point scale (0-2), providing a score with a range from 0 to10 (see Table 1 for detailed descriptions of quality assessment). Studies obtaining a total score above two-thirds of the maximum score (ie, >7) were considered high quality studies.

Table 1.

Study assessment criteria for scoring.

Aspect Scoring
Study design Which design did the study have? 2=controlled trial, 1=pre-post, 0=case study or unclear.
Randomization Were participants randomized to conditions (depending on design)? 2=yes, 0=no.
Report of statistics Are relevant statistics reported? 2=mean and standard deviation for outcome measures, effect sizes, and P values for significant differences are reported, 1=is lacking any of these, 0=lack several of these.
Sample size Was the sample size adequate to detect effect sizes of d=0.35? 2=264 or more, 1=more than 132, 0=132 or less
Nonblended control group Did the design involve a nonblended active control group? 2=nonblended control group with same number of sessions, 1=active control group without assured same number of sessions, 0=no active control group
Open in a new tab

Results

Study selection

See Figure 1 for a flowchart of included and excluded studies. In total, we identified 64 publications describing 44 studies. Eight of the included studies were study protocols that had not yet published results in November 2016.

Figure 1.

Figure 1

Open in a new tab

Flowchart of included and excluded studies.

Of the included studies, 27 were randomized controlled trials (RCTs), 4 were non-RCTs, 5 were pre-post studies without a control group, 4 were case studies, 3 were preliminary evaluation or acceptability studies, and 1 was a qualitative study. See Multimedia Appendix 1 for detailed study characteristics.

Of the 44 studies, 8 were rated high quality studies (18%). A control group was involved in 31 (70%, 18/44) of the studies. Twelve studies (27%, 12/44) involved a nonblended active control group with the same number of sessions as the intervention group. Only 6 studies (14%, 6/44) involved a sample of 264 or more. All relevant statistics, including effect sizes, were reported by 17 studies (39%, 17/44). See Multimedia Appendix 3 for all quality assessment scores.

Most studies were conducted or planned in the United States (n=12), followed by the Netherlands (n=9) and Germany (n=8). Six studies were conducted in Australia, four in Norway, three in the United Kingdom, and two in Sweden.

The Internet part of the studies’ blended interventions generally used Web-based programs with modules combining techniques such as cognitive, behavioral, and/or emotion-focused interventions, some of them with email support. However, there were two exceptions. One study [23] reports on using emails and Internet sites on psychoeducation and other issues to improve organization and to assist with common problems related to depression. The other study [44] reports offering weekly Internet group chats with a therapist (in open groups of 8-10 participants) to focus on problems arising in readjusting to everyday life after inpatient treatment.

Disorders Addressed

Of the included 44 studies, 20 studies focused on treating depression only, eight focused on anxiety disorders only, and eight focused on substance abuse only. One study described the treatment of comorbid depression and substance abuse; three studies treated both depression and anxiety. The remaining four studies had a transdiagnostic concept involving depression, anxiety, and other mental disorders. See Multimedia Appendix 1 for details on disorders addressed.

Concept of Blended Care and Stage of Treatment

Following the study selection process, we clustered the studies into the following types of blended care (Multimedia Appendix 1):

  1. Integrated blended interventions with face-to-face focus: These blended interventions are based on an face-to-face intervention that is complemented or partly replaced by an Internet intervention; face-to-face and Internet-based elements are provided within the same period.

  2. Integrated blended interventions with Internet focus: These blended interventions are based on Internet interventions that are partly replaced or complemented by face-to-face sessions; face-to-face and Internet-based elements are provided within the same period.

  3. Sequential blended interventions with Internet, then face-to-face: These blended interventions arrange the Internet intervention part before the face-to-face treatment, such as within stepped care.

  4. Sequential blended interventions with face-to-face, then Internet: These blended interventions arrange the Internet intervention part after the face-to-face treatment as in an aftercare program.

The majority of studies (n=29) used a concept of integrated blended intervention, with face-to-face and Internet-based elements being provided within the same period. Among those, 18 studies focused on the face-to-face intervention, considering the Internet intervention as a replacement of some of the face-to-face sessions or as an adjunct, whereas 11 studies focused on the Internet intervention as the basis of treatment where the face-to-face sessions served as an adjunct, for example, for increasing adherence to the Internet-based modules.

The remaining 15 studies presented sequential blended interventions, arranging the Internet intervention part before or following the face-to-face treatment. Nine of them placed the Internet intervention before the face-to-face treatment either as part of a stepped or matched care program (n=4) or for bridging waiting time for referrals on waiting lists for face-to-face psychotherapy (n=5). Six studies placed the Internet intervention after the face-to-face treatment as an aftercare concept.

Aims

The studies’ aims for choosing to use a blended intervention can be classified by the concepts that were used (see Multimedia Appendix 1 for details). As some studies mentioned various aims, we clustered the interventions by their main objective. See Figure 2 for a summary of aims of blended therapy.

Figure 2.

Figure 2

Open in a new tab

Aims of blended therapy.

Integrated Blended Interventions With Face-to-Face (F2F) Focus

Among the 18 integrated blended interventions with face-to-face focus, seven aimed at delegating some elements of face-to-face therapy to Internet-based cognitive behavioral therapy (iCBT) and thereby, saving clinician time and reducing overall costs [15,51,54,58,62,69,75]. Nine of them aimed at supporting face-to-face therapy by delivering additional Internet elements and thereby increasing effectiveness of face-to-face psychotherapy [23-25,37,40,55,60,61,77,78]. Two studies stated as their aim integrating Internet elements with face-to-face psychotherapy to establish a proactive and long-term approach to the management of chronic conditions, thus providing long-term support for patients with chronic or recurrent mental diseases beyond the acute phase of face-to-face treatment [59,68].

Integrated Blended Interventions With Internet Focus

Five of the 11 integrated blended interventions with an Internet focus aimed at improving the delivery of evidence-based treatment in primary care [31,48,50,65,72], for instance, by assisting GPs in providing evidence-based mental health care programs [65]. Three of the studies aimed at integrating face-to-face sessions to maximize the effectiveness of iCBT [19,64,70]. Other aims of these studies were to offer an Internet intervention to all participants with complementary face-to-face sessions as needed by the individual participant and as such to increase flexibility to meet the needs of different clients concerning face-to-face support [79], or, through face-to-face support, to motivate participants to persist with iCBT [45,74].

Sequential Blended Interventions With Internet, Then F2F

The nine sequential blended interventions that started with the Internet intervention either aimed at bridging waiting time (n=5) with iCBT until face-to-face therapy started [53,57,67,73,80] or worked with a blended stepped care concept, aiming at delivering low-threshold iCBT as an early step and thereby reducing costs for subsequent face-to-facepsychotherapy (n=4) (by reducing either the number of v sessions or the number of patients treated in the subsequent face-to-facephase) [20,26,46,47].

Sequential Blended Interventions With F2F, Then Internet

Five of the six sequential blended interventions that started with the face-to-face intervention were designed as aftercare programs aiming at maintaining therapeutic benefits of face-to-faceresidential or inpatient psychotherapy through subsequent iCBT [22,41,44,56,76]. Another study aimed to monitor patient progress beyond the acute phase of face-to-face treatment for a long time and to maintain the therapeutic relationship in the absence of face-to-face contacts in a remote setting [71].

Outcome

See Multimedia Appendix 2 for details. Given the variety of different study types (eg, study protocols, case studies, and qualitative studies) and that outcome measures, study designs, and aims differ substantially, it is not currently possible to summarize the effects using meta-analytic techniques.

Of the 44 identified studies, eight were study protocols and 36 had been completed. Among the 36 completed studies, 23 involved a control group. Out of the 23 completed studies with a control group, four studies compared the blended intervention with no treatment, for instance, a waiting-list group [19,41,44,48]. Nine studies compared the blended intervention group with a nonblended face-to-face intervention with an equal amount or more sessions [24,32,50,53-55,57,62,75], six studies compared the blended intervention with treatment-as-usual without controlling the number of sessions [20,26,32,37,40], and two studies compared the blended intervention with a nonblended Internet intervention [64,65]. The remaining two studies involved both Internet-based and face-to-face nonblended control groups [55,70].

Among the eight study protocols, two studies compared the blended intervention with TAU without controlling for the number of sessions [46,51,59], whereas four studies compared the blended intervention-group with a nonblended face-to-face intervention with an equal number or more sessions [15,53,60,69]. One study compared the blended intervention (in that case, aftercare after inpatient treatment) with a placebo (inpatient treatment plus access after discharge to an Internet platform with information on depression [76]), and one study compared it with no treatment [67].

Although cost-effectiveness or cost-savings was in some way considered by almost all 44 studies, only three of the completed studies elaborated on it and evaluated potential cost-effectiveness or cost-savings [33,52,81]. However, three additional study protocols claimed to be planning to evaluate cost-effectiveness [15,46,51,69]. See Figure 3 for a summary of aims of blended therapy in completed studies.

Figure 3.

Figure 3

Open in a new tab

Aims of blended therapy of studies with published outcome.

Integrated Blended Interventions With F2F Focus

Of the six studies that were delegating some elements of face-to-face therapy to iCBT, three studies were able to show that, by doing that, 50% to 86 % of clinician time could be saved without reducing the therapeutic outcome of depression and anxiety treatment [54,62,75]. The other three were study protocols [15,51,69].

Six of the nine studies aiming at supporting face-to-face therapy by delivering Internet elements were able to show that adding Internet elements can lead to lower dropout rates and/or greater abstinence rates of patients with substance abuse compared with stand-alone face-to-face interventions [24,40,55,77]. One study had a pre-post design [61] and showed that its blended concept led to a reduction in symptomatology that was maintained for 12 months. One was a case study where a patient diagnosed with depression as well as her therapist were able to make creative use of Web-based resources for purposes such as psychoeducation or job search [23]. One was a study protocol [60].

One study aiming at establishing a proactive and long-term approach to the management of chronic mental diseases beyond the acute phase of face-to-face treatment [68] showed large and significant pre-post effect sizes. One study with a similar aim [59] showed that an Internet-delivered adjunct to TAU may reduce the number of unwell weeks in patients with recurrent depression compared with TAU and thus, reduce the lifelong burden of depression.

Integrated Blended Interventions With Internet Focus

Regarding the aim of integrating face-to-face sessions into Internet interventions to maximize effectiveness of iCBT, Sethi et al [70] found the blended intervention to be superior in reducing symptoms of depression, anxiety, and/or automatic negative thoughts in comparison with three control groups: nonblended Internet intervention, nonblended face-to-faceintervention, and no intervention. One study treating depression [64] did not find a difference between the blended and the nonblended intervention, whereas a third study did not have a nonblended control group but found the blended intervention to be superior compared with a waiting-list control group [19].

Using blended interventions to improve the delivery of evidence-based treatment in primary care has been successful in acceptability and in reducing symptomatology [31,48,50,65,72]. However, two controlled studies [50,65] could not show a superiority of the blended intervention over an active nonblended control group.

In a study aiming at increasing the flexibility of an Internet intervention by offering complementary face-to-face sessions as needed by the individual participant [79], the average number of face-to-face sessions participants needed was equal to or less than 3.7, with the program still substantially and significantly reducing depression symptomatology. This was a pilot study without a control group, yet the authors cite a study that treated participants with similar pre- and post-Beck depression inventory scores as their study [82]. The other study’s participants needed an average of 11.6 face-to-face sessions for a similar symptom reduction, and the authors therefore claim that their treatment needs about eight fewer individual sessions per client.

A study aiming at motivating participants to persist with iCBT through face-to-face support [74] used a qualitative design. It found that for persistence in a blended treatment, acknowledgment may be related to flexibility and feedback from a qualified therapist in the face-to-face consultations beneath personal resources such as a sense of belonging toward partners, family, and friends. A case study with a similar aim [45] showed good adherence and symptom remission in a woman suffering from antepartum depression, who was thus able to avoid antidepressant use during pregnancy.

Sequential Blended Interventions With Internet, Then F2F

Of the five studies aiming at bridging waiting time with iCBT until face-to-face therapy starts, two studies [53,57] showed significant between-group effect sizes between the iCBT waiting-list group and the control waiting-list group. One study failed to show a superiority of the iCBT waiting-list group compared with a control condition with participants who remained on the wait-list with a self-help booklet [83]. One of the studies showed in a pre-post design that the Internet intervention could substantially and significantly reduce symptomatology [73]. The remaining study with this aim [67] is a study protocol.

Two of the completed studies working with a blended stepped care concept did not find a significant superiority of the blended intervention compared with face-to-face [47] or TAU [20]. Haug et al [47] even found a significantly better outcome of the control group (nonblended f2f) in one of the outcome measures. Neither Haug et al nor Braamse et al [20] reported results on cost-saving or cost-effectiveness; however, one study protocol with a stepped care concept mentioned planning to evaluate cost-effectiveness [46].

Sequential Blended Interventions With F2F, Then Internet

Two of the five studies with Internet intervention designed as maintenance treatment found a substantially and significantly lower relapse rate compared with access to TAU groups [41,44]. One further study was a case study reporting how iCBT aftercare substantially reduced depression, leading to remission after discharge from inpatient treatment [22]. One study with an empirical correlational design and without a control group showed a significant relationship between the number of iCBT modules accessed and substance abuse reduction in the year following inpatient treatment when controlling for motivation, self-efficacy, and pretreatment substance abuse [56]. One study was a study protocol [76].

One case study aiming at long-term monitoring of patient progress beyond the acute phase of face-to-face treatment, as well as maintaining the therapeutic relationship in the absence of face-to-face contacts in a remote setting [71], reported continuing therapeutic contacts and a reduction from very severe depression symptoms to remission within 46 weeks.

Discussion

Principal Findings

This study has shown that, in the past few years, a growing number of blended interventions that combine Internet and face-to-face interventions have been developed for common mental health problems. The interventions we found have different concepts and various aims. First results are encouraging and suggest that, compared with stand-alone face-to-face interventions, blended therapy may save clinician time without reducing therapy outcome, can lead to lower dropout rates and/or greater abstinence rates of patients with substance abuse, may help maintain effects of inpatient therapy, and may even increase the effects of psychotherapy, although results are mixed and more research is clearly needed.

Compared with the field of both Internet-based stand-alone treatment and face-to-face interventions, the field of blended interventions is, however, under development and still small. Most aims of the interventions stated in the studies have not been evaluated rigorously. For instance, only 19 out of 36 completed studies were RCTs, and only eight of them ensured comparability by involving a nonblended intervention control group with the same number of sessions [27,50,57,62,70,75,78,83]. Only eight studies were considered high quality studies, and only six studies involved a control group big enough to detect effect sizes of d=0.35 or less. Also, the cost-effectiveness of blended treatments compared with face-to-face psychotherapy has only been evaluated in three out of the 36 completed studies. However, this issue is focused upon in current large research projects [46,84].

Several questions remain. For instance, not much is known about the optimal ratio of Internet and face-to-facesessions that would allow costs to be minimized while maintaining or increasing effectiveness. Only one study in our review is moving toward answering this question: In the study by Jacmon et al [79], clients received face-to-face sessions as wished as an adjunct to iCBT. Also, clients were actively invited by therapists to undertake face-to-face contacts if a depression measure indicated that depression continued at clinical levels. Jacmon et al reported that this intervention produced effects at least as large as a completely face-to-face treatment, with an average of about eight fewer individual sessions per client in comparison with a similar nonblended face-to-face treatment in another study. However, the findings of this pilot study cannot be generalized because of its small sample size and lack of a control group. Another open question is which elements of face-to-face therapy can most suitably be delegated to the Internet. Possibly, elements that do not need an intensive dialogue between client and therapist, such as psychoeducation, may most easily be delegated to the Internet [16]. Also, it remains unclear who would benefit most from which relationship between face-to-face and Internet modules. For instance, patients who have difficulty in expressing all of their thoughts and feelings in writing (eg, because of lack of introspection or because they are not conscious of their own role in maintaining their disorder) might need more elements of face-to-face intervention psychotherapy. This might also be the case for patients with severe, chronic, or personality disorders. A helpful tool to set up a personalized blended treatment taking the patients’ characteristics into account might be the recently developed “Fit for Blended Care” instrument [17].

Moreover, although a number of studies stated that increasing the effectiveness of face-to-face psychotherapy is an aim of the blending of treatments with Internet options, evidence for this hypothesis is limited to four studies with an integrated concept [24,25,32,70]. Such an assumption is, however, supported by a recent meta-analysis by Lindhiem and colleagues [85]. On the basis of 10 RCTs, they found that, in comparison with strictly on-site interventions, psychological interventions were considerably more effective for a range of conditions when behavior changes between face-to-face sessions had been supported by a mobile component, such as short message service (SMS; d=0.27). There is also some evidence that telephone-based interventions may help maintain advances achieved during outpatient treatment or increase effectiveness of face-to-face interventions [86,87]. Nevertheless, more research is clearly needed to conclude whether the effectiveness of psychotherapy can potentially be increased by using blended treatments. Besides the identified advantages of blended interventions, for example, saving clinician time, improving success rates in the reduction of symptomatology, and helping to prevent relapse after face-to-face therapy, these interventions might possibly have limitations compared with face-to-face interventions. For instance, in the study by Marks et al [62], dropout was substantially higher in the mainly Internet-based group compared with the face-to-face control group (43% if mainly computer-guided and 24% if entirely clinician-guided). On the other hand, other studies have reported similar [50,75] or even lower [24] dropout rates of a blended compared with a nonblended intervention. Yet, the questions of blended interventions’ negative effects and potential losses compared with nonblended interventions remain to be answered, just as much as the one about potential negative effects of Internet-based treatments [12].

An interesting question we encountered is what blended treatments exactly entail. The identified studies in our review that explicitly use the term “blended” are the more recently published ones and describe integrated blended treatments, that is, combined treatments that provide face-to-face and Internet-based elements within the same period. Although there is currently no clear definition, this description might be an implicit understanding of blended interventions that is more limiting than our definition. We consider that blended interventions should also include the combination of the Internet-based interventions arranged before or after the face-to-face treatments following clear rules and procedures, such as within stepped care or aftercare that directly follows the acute phase treatment. In light of the numerous studies describing sequential blended interventions, we decided to use this wider definition.

Limitations

This study has some limitations. As within every systematic review, the risk of selection bias when including relevant studies needs to be considered. However, our use of independent ratings by two of the authors worked against this bias. In addition, publication bias needs to be considered. We did not contact authors for additional data or additional studies, which would have automatically limited the number of studies that could be included. Also, we only reviewed bibliographies from included studies, so we possibly missed studies that were cited in papers that we reviewed but did not include. Furthermore, the types of studies we included were heterogeneous (for instance, we included study protocols). It is possible that more narrow inclusion criteria (eg, randomized controlled studies with nonblended active control groups only) would have produced more information about the effectiveness of blended interventions. However, the number of such studies in the field is yet very small, and future studies are needed to explore whether blended treatments can, for example, be superior compared with nonblended treatments with regard to effect sizes or lower costs.

For further research, it would be of interest to explore effectiveness and cost-effectiveness of blended concepts, especially concerning the optimal balance of face-to-face and Internet interventions. Information on this aspect would help determine which therapy with which theoretical foundation (such as cognitive behavioral therapy or psychoanalysis) is feasible for blended interventions, as well as where the optimal balance of therapy modules lies for individual patients in light of the type and severity of disorder, state of motivation, ability of introspection, and other variables such as age, gender, and computer skills.

Conclusions

To conclude, we have found that several studies have shown that blended interventions are feasible and effective compared with no treatment controls. There are many different kinds of blended concepts that, in every phase of treatment, may offer added value concerning either effectiveness or cost-effectiveness. However, to evaluate the actual benefit of blended concepts for mental health care, more RCTs on effectiveness and cost-effectiveness compared with traditional nonblended psychotherapy are required. Thus, more research is needed, especially concerning disorders for which blended interventions are particularly effective, the amount of face-to-face contact that is needed, and the parts of therapy that can be delegated to the Internet.

Abbreviations

f2f

face-to-face

RCTs

randomized controlled trials

Multimedia Appendix 1

Previous psychological research combining Internet and face-to-face psychotherapy.

jmir_v19i9e306_app1.pdf (78.7KB, pdf)
Multimedia Appendix 2

Outcome of previous psychological research combining Internet and face-to-face (f2f) psychotherapy.

jmir_v19i9e306_app2.pdf (87.8KB, pdf)
Multimedia Appendix 3

Study quality assessment of blended treatments.

jmir_v19i9e306_app3.pdf (42.4KB, pdf)

Footnotes

Conflicts of Interest: None declared.

References

  • 1.Andersson G, Topooco N, Havik O, Nordgreen T. Internet-supported versus face-to-face cognitive behavior therapy for depression. Expert Rev Neurother. 2016;16(1):55–60. doi: 10.1586/14737175.2015.1125783. [DOI] [PubMed] [Google Scholar]
  • 2.Ebert DD, Zarski AC, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, Riper H. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One. 2015;10(3):e0119895. doi: 10.1371/journal.pone.0119895. http://dx.plos.org/10.1371/journal.pone.0119895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Mayo-Wilson E, Montgomery P. Media-delivered cognitive behavioural therapy and behavioural therapy (self-help) for anxiety disorders in adults. Cochrane Database Syst Rev. 2013;(9):1–48. doi: 10.1002/14651858.CD005330.pub4. [DOI] [PubMed] [Google Scholar]
  • 4.Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012 Jun;32(4):329–42. doi: 10.1016/j.cpr.2012.02.004. [DOI] [PubMed] [Google Scholar]
  • 5.Riper H, Blankers M, Hadiwijaya H, Cunningham J, Clarke S, Wiers R, Ebert D, Cuijpers P. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One. 2014;9(6):e99912. doi: 10.1371/journal.pone.0099912. http://dx.plos.org/10.1371/journal.pone.0099912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Königbauer J, Letsch J, Ebler P, Ebert DD, Baumeister H. Internet- and mobile-based depression interventions for people with diagnosed depression: a systematic review and meta-analysis. J Affect Disord. 2017 Jul 10;223:28–40. doi: 10.1016/j.jad.2017.07.021. [DOI] [PubMed] [Google Scholar]
  • 7.Cuijpers P, van SA, Andersson G. Internet-administered cognitive behavior therapy for health problems: a systematic review. J Behav Med. 2008 Apr;31(2):169–77. doi: 10.1007/s10865-007-9144-1. http://europepmc.org/abstract/MED/18165893. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Ebert DD, Berking M, Thiart H, Riper H, Laferton JA, Cuijpers P, Sieland B, Lehr D. Restoring depleted resources: Efficacy and mechanisms of change of an internet-based unguided recovery training for better sleep and psychological detachment from work. Health Psychol. 2015 Dec;34S:1240–51. doi: 10.1037/hea0000277. [DOI] [PubMed] [Google Scholar]
  • 9.Berger T, Andersson G. [Internet-based psychotherapies: characteristics and empirical evidence] Psychother Psychosom Med Psychol. 2009;59(3-4):159–66; quiz 167. doi: 10.1055/s-0028-1090162. [DOI] [PubMed] [Google Scholar]
  • 10.Boettcher J, Rozental A, Andersson G, Carlbring P. Side effects in internet-based interventions for social anxiety disorder. Internet Interv. 2014 Mar;1(1):3–11. doi: 10.1016/j.invent.2014.02.002. [DOI] [Google Scholar]
  • 11.Rozental A, Boettcher J, Andersson G, Schmidt B, Carlbring P. Negative effects of internet interventions: a qualitative content analysis of patients' experiences with treatments delivered online. Cogn Behav Ther. 2015;44(3):223–36. doi: 10.1080/16506073.2015.1008033. [DOI] [PubMed] [Google Scholar]
  • 12.Rozental A, Andersson G, Boettcher J, Ebert DD, Cuijpers P, Knaevelsrud C, Ljótsson B, Kaldo V, Titov N, Carlbring P. Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interv. 2014 Mar;1(1):12–9. doi: 10.1016/j.invent.2014.02.001. [DOI] [Google Scholar]
  • 13.Ebert DD, Donkin L, Andersson G, Andrews G, Berger T, Carlbring P, Rozenthal A, Choi I, Laferton JA, Johansson R, Kleiboer A, Lange A, Lehr D, Reins JA, Funk B, Newby J, Perini S, Riper H, Ruwaard J, Sheeber L, Snoek FJ, Titov N, Ünlü IB, van BK, Vernmark K, van SA, Warmerdam L, Salsman N, Cuijpers P. Does internet-based guided-self-help for depression cause harm? an individual participant data meta-analysis on deterioration rates and its moderators in randomized controlled trials. Psychol Med. 2016 Oct;46(13):2679–93. doi: 10.1017/S0033291716001562. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Moritz S, Schröder J, Meyer B, Hauschildt M. The more it is needed, the less it is wanted: attitudes toward face-to-face intervention among depressed patients undergoing online treatment. Depress Anxiety. 2013 Feb;30(2):157–67. doi: 10.1002/da.21988. [DOI] [PubMed] [Google Scholar]
  • 15.Kooistra LC, Wiersma JE, Ruwaard J, van Oppen P, Smit F, Lokkerbol J, Cuijpers P, Riper H. Blended vs. face-to-face cognitive behavioural treatment for major depression in specialized mental health care: study protocol of a randomized controlled cost-effectiveness trial. BMC Psychiatry. 2014 Oct 18;14:290. doi: 10.1186/s12888-014-0290-z. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-014-0290-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.van der Vaart R, Witting M, Riper H, Kooistra L, Bohlmeijer ST, van Gemert-Pijnen LJ. Blending online therapy into regular face-to-face therapy for depression: content, ratio and preconditions according to patients and therapists using a Delphi study. BMC Psychiatry. 2014 Dec 14;14:355. doi: 10.1186/s12888-014-0355-z. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-014-0355-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Wentzel J, van der Vaart R, Bohlmeijer ET, van der Vaart R. Mixing online and face-to-face therapy: how to benefit from blended care in mental health care. JMIR Ment Health. 2016 Feb 09;3(1):e9. doi: 10.2196/mental.4534. http://mental.jmir.org/2016/1/e9/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Cuijpers P, Huibers M, Ebert DD, Koole SL, Andersson G. How much psychotherapy is needed to treat depression? a metaregression analysis. J Affect Disord. 2013 Jul;149(1-3):1–13. doi: 10.1016/j.jad.2013.02.030. [DOI] [PubMed] [Google Scholar]
  • 19.Andersson G, Carlbring P, Holmström A, Sparthan E, Furmark T, Nilsson-Ihrfelt E, Buhrman M, Ekselius L. Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial. J Consult Clin Psychol. 2006 Aug;74(4):677–86. doi: 10.1037/0022-006X.74.4.677. [DOI] [PubMed] [Google Scholar]
  • 20.Braamse AM, van MB, Visser OJ, Boenink AD, Cuijpers P, Eeltink CE, Hoogendoorn AW, van MK, van OP, Huijgens PC, Beekman AT, Dekker J. A randomized clinical trial on the effectiveness of an intervention to treat psychological distress and improve quality of life after autologous stem cell transplantation. Ann Hematol. 2016 Jan;95(1):105–14. doi: 10.1007/s00277-015-2509-6. http://europepmc.org/abstract/MED/26420062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Braamse AM, van Meijel B, Visser OJ, Boenink AD, Cuijpers P, Eeltink CE, Hoogendoorn AW, van Marwijk Kooy M, van Oppen P, Huijgens PC, Beekman AT, Dekker J. A randomized clinical trial on the effectiveness of an intervention to treat psychological distress and improve quality of life after autologous stem cell transplantation. Ann Hematol. 2016 Jan;95(1):105–14. doi: 10.1007/s00277-015-2509-6. http://europepmc.org/abstract/MED/26420062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Brakemeier E, Marchner J, Gutgsell S, Engel V, Radtke M, Tuschen-Caffier B, Normann C, Berking M. CBASP@home: ein internetbasiertes situationsanalysen- training zur stabilisierung des therapieerfolgs nach stationärer therapie für chronisch depressive patienten. Verhaltenstherapie. 2013 Aug;23:190–203. doi: 10.1159/00035481. [DOI] [Google Scholar]
  • 23.Callan JA, Howland RH, Puskar K. Using computers and the Internet for psychiatric nursing intervention. J Psychosoc Nurs Ment Health Serv. 2009 Jan;47(1):13–4. doi: 10.3928/02793695-20090101-03. [DOI] [PubMed] [Google Scholar]
  • 24.Campbell AN, Nunes EV, Matthews AG, Stitzer M, Miele GM, Polsky D, Turrigiano E, Walters S, McClure EA, Kyle TL, Wahle A, Van VP, Goldman B, Babcock D, Stabile PQ, Winhusen T, Ghitza UE. Internet-delivered treatment for substance abuse: a multisite randomized controlled trial. Am J Psychiatry. 2014 Jun;171(6):683–90. doi: 10.1176/appi.ajp.2014.13081055. http://europepmc.org/abstract/MED/24700332. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Cochran G, Stitzer M, Campbell AN, Hu MC, Vandrey R, Nunes EV. Web-based treatment for substance use disorders: differential effects by primary substance. Addict Behav. 2015 Jun;45:191–4. doi: 10.1016/j.addbeh.2015.02.002. http://europepmc.org/abstract/MED/25697725. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.van Straten A, Seekles W, van't Veer-Tazelaar NJ, Beekman AT, Cuijpers P. Stepped care for depression in primary care: what should be offered and how? Med J Aust. 2010 Jun 07;192(11 Suppl):S36–9. doi: 10.5694/j.1326-5377.2010.tb03691.x. [DOI] [PubMed] [Google Scholar]
  • 27.Campbell AN, Nunes EV, Pavlicova M, Hatch-Maillette M, Hu MC, Bailey GL, Sugarman DE, Miele GM, Rieckmann T, Shores-Wilson K, Turrigiano E, Greenfield SF. Gender-based outcomes and acceptability of a computer-assisted psychosocial intervention for substance use disorders. J Subst Abuse Treat. 2015 Jun;53:9–15. doi: 10.1016/j.jsat.2014.12.006. http://europepmc.org/abstract/MED/25613105. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Cunningham C, Stitzer M, Campbell AN, Pavlicova M, Hu MC, Nunes EV. Contingency management abstinence incentives: cost and implications for treatment tailoring. J Subst Abuse Treat. 2017 Jan;72:134–9. doi: 10.1016/j.jsat.2015.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Tofighi B, Campbell AN, Pavlicova M, Hu MC, Lee JD, Nunes EV. Recent internet use and associations with clinical outcomes among patients entering addiction treatment involved in a web-delivered psychosocial intervention study. J Urban Health. 2016 Oct;93(5):871–83. doi: 10.1007/s11524-016-0077-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Campbell AN, Nunes EV, Miele GM, Matthews A, Polsky D, Ghitza UE, Turrigiano E, Bailey GL, VanVeldhuisen P, Chapdelaine R, Froias A, Stitzer ML, Carroll KM, Winhusen T, Clingerman S, Perez L, McClure E, Goldman B, Crowell AR. Design and methodological considerations of an effectiveness trial of a computer-assisted intervention: an example from the NIDA Clinical Trials Network. Contemp Clin Trials. 2012 Mar;33(2):386–95. doi: 10.1016/j.cct.2011.11.001. http://europepmc.org/abstract/MED/22085803. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Campbell AN, Turrigiano E, Moore M, Miele GM, Rieckmann T, Hu MC, Kropp F, Ringor-Carty R, Nunes EV. Acceptability of a web-based community reinforcement approach for substance use disorders with treatment-seeking American Indians/Alaska Natives. Community Ment Health J. 2015 May;51(4):393–403. doi: 10.1007/s10597-014-9764-1. http://europepmc.org/abstract/MED/25022913. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Carroll KM, Ball SA, Martino S, Nich C, Babuscio TA, Nuro KF, Gordon MA, Portnoy GA, Rounsaville BJ. Computer-assisted delivery of cognitive-behavioral therapy for addiction: a randomized trial of CBT4CBT. Am J Psychiatry. 2008 Jul;165(7):881–8. doi: 10.1176/appi.ajp.2008.07111835. http://europepmc.org/abstract/MED/18450927. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Olmstead TA, Ostrow CD, Carroll KM. Cost-effectiveness of computer-assisted training in cognitive-behavioral therapy as an adjunct to standard care for addiction. Drug Alcohol Depend. 2010 Aug 01;110(3):200–7. doi: 10.1016/j.drugalcdep.2010.02.022. http://europepmc.org/abstract/MED/20392575. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Carroll KM, Ball SA, Martino S, Nich C, Babuscio TA, Rounsaville BJ. Enduring effects of a computer-assisted training program for cognitive behavioral therapy: a 6-month follow-up of CBT4CBT. Drug Alcohol Depend. 2009 Feb 01;100(1-2):178–81. doi: 10.1016/j.drugalcdep.2008.09.015. http://europepmc.org/abstract/MED/19041197. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Carroll KM, Kiluk BD, Nich C, Babuscio TA, Brewer JA, Potenza MN, Ball SA, Martino S, Rounsaville BJ, Lejuez CW. Cognitive function and treatment response in a randomized clinical trial of computer-based training in cognitive-behavioral therapy. Subst Use Misuse. 2011;46(1):23–34. doi: 10.3109/10826084.2011.521069. http://europepmc.org/abstract/MED/21190403. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Sugarman DE, Nich C, Carroll KM. Coping strategy use following computerized cognitive-behavioral therapy for substance use disorders. Psychol Addict Behav. 2010 Dec;24(4):689–95. doi: 10.1037/a0021584. http://europepmc.org/abstract/MED/21198228. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Carroll KM, Kiluk BD, Nich C, Gordon MA, Portnoy GA, Marino DR, Ball SA. Computer-assisted delivery of cognitive-behavioral therapy: efficacy and durability of CBT4CBT among cocaine-dependent individuals maintained on methadone. Am J Psychiatry. 2014 Apr;171(4):436–44. doi: 10.1176/appi.ajp.2013.13070987. http://europepmc.org/abstract/MED/24577287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Carroll KM, Herman A, DeVito EE, Frankforter TL, Potenza MN, Sofuoglu M. Catehol-o-methyltransferase gene Val158met polymorphism as a potential predictor of response to computer-assisted delivery of cognitive-behavioral therapy among cocaine-dependent individuals: preliminary findings from a randomized controlled trial. Am J Addict. 2015 Aug;24(5):443–51. doi: 10.1111/ajad.12238. http://europepmc.org/abstract/MED/25930952. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Morie KP, Nich C, Hunkele K, Potenza MN, Carroll KM. Alexithymia level and response to computer-based training in cognitive behavioral therapy among cocaine-dependent methadone maintained individuals. Drug Alcohol Depend. 2015 Jul 01;152:157–63. doi: 10.1016/j.drugalcdep.2015.04.004. http://europepmc.org/abstract/MED/25982006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Christensen DR, Landes RD, Jackson L, Marsch LA, Mancino MJ, Chopra MP, Bickel WK. Adding an Internet-delivered treatment to an efficacious treatment package for opioid dependence. J Consult Clin Psychol. 2014 Dec;82(6):964–72. doi: 10.1037/a0037496. http://europepmc.org/abstract/MED/25090043. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Ebert D, Tarnowski T, Gollwitzer M, Sieland B, Berking M. A transdiagnostic internet-based maintenance treatment enhances the stability of outcome after inpatient cognitive behavioral therapy: a randomized controlled trial. Psychother Psychosom. 2013;82(4):246–56. doi: 10.1159/000345967. [DOI] [PubMed] [Google Scholar]
  • 42.Ebert DD, Gollwitzer M, Riper H, Cuijpers P, Baumeister H, Berking M. For whom does it work? moderators of outcome on the effect of a transdiagnostic internet-based maintenance treatment after inpatient psychotherapy: randomized controlled trial. J Med Internet Res. 2013 Oct 10;15(10):e191. doi: 10.2196/jmir.2511. http://www.jmir.org/2013/10/e191/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Ebert DD, Hannig W, Tarnowski T, Sieland B, Götzky B, Berking M. [Web-based rehabilitation aftercare following inpatient psychosomatic treatment] Rehabilitation (Stuttg) 2013 Jun;52(3):164–72. doi: 10.1055/s-0033-1345191. [DOI] [PubMed] [Google Scholar]
  • 44.Golkaramnay V, Bauer S, Haug S, Wolf M, Kordy H. The exploration of the effectiveness of group therapy through an Internet chat as aftercare: a controlled naturalistic study. Psychother Psychosom. 2007;76(4):219–25. doi: 10.1159/000101500. [DOI] [PubMed] [Google Scholar]
  • 45.Hantsoo L, Epperson CN, Thase ME, Kim DR. Antepartum depression: treatment with computer-assisted cognitive-behavioral therapy. Am J Psychiatry. 2013 Aug;170(8):929–30. doi: 10.1176/appi.ajp.2013.13020254. http://europepmc.org/abstract/MED/23903340. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Härter M, Heddaeus D, Steinmann M, Schreiber R, Brettschneider C, König HH, Watzke B. [Collaborative and stepped care for depression: Development of a model project within the hamburg network for mental health (psychenet.de)] Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2015 Apr;58(4-5):420–9. doi: 10.1007/s00103-015-2124-7. [DOI] [PubMed] [Google Scholar]
  • 47.Haug T, Nordgreen T, Öst LG, Kvale G, Tangen T, Andersson G, Carlbring P, Heiervang ER, Havik OE. Stepped care versus face-to-face cognitive behavior therapy for panic disorder and social anxiety disorder: predictors and moderators of outcome. Behav Res Ther. 2015 Aug;71:76–89. doi: 10.1016/j.brat.2015.06.002. [DOI] [PubMed] [Google Scholar]
  • 48.Høifødt RS, Lillevoll KR, Griffiths KM, Wilsgaard T, Eisemann M, Waterloo K, Kolstrup N. The clinical effectiveness of web-based cognitive behavioral therapy with face-to-face therapist support for depressed primary care patients: randomized controlled trial. J Med Internet Res. 2013 Aug 05;15(8):e153. doi: 10.2196/jmir.2714. http://www.jmir.org/2013/8/e153/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Høifødt RS, Mittner M, Lillevoll K, Katla SK, Kolstrup N, Eisemann M, Friborg O, Waterloo K. Predictors of response to web-based cognitive behavioral therapy with high-intensity face-to-face therapist guidance for depression: a bayesian analysis. J Med Internet Res. 2015 Sep 02;17(9):e197. doi: 10.2196/jmir.4351. http://www.jmir.org/2015/9/e197/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Kay-Lambkin F, Baker A, Lewin T, Carr V. Acceptability of a clinician-assisted computerized psychological intervention for comorbid mental health and substance use problems: treatment adherence data from a randomized controlled trial. J Med Internet Res. 2011 Jan 27;13(1):e11. doi: 10.2196/jmir.1522. http://www.jmir.org/2011/1/e11/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Kemmeren LL, van Schaik DJF, Riper H, Kleiboer AM, Bosmans JE, Smit JH. Effectiveness of blended depression treatment for adults in specialised mental healthcare: study protocol for a randomised controlled trial. BMC Psychiatry. 2016 Apr 21;16:113. doi: 10.1186/s12888-016-0818-5. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-016-0818-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Kolovos S, Kenter RM, Bosmans JE, Beekman AT, Cuijpers P, Kok RN, van SA. Economic evaluation of internet-based problem-solving guided self-help treatment in comparison with enhanced usual care for depressed outpatients waiting for face-to-face treatment: A randomized controlled trial. J Affect Disord. 2016 Aug;200:284–92. doi: 10.1016/j.jad.2016.04.025. [DOI] [PubMed] [Google Scholar]
  • 53.Kok RN, van Straten A, Beekman AT, Cuijpers P. Short-term effectiveness of web-based guided self-help for phobic outpatients: randomized controlled trial. J Med Internet Res. 2014 Sep 29;16(9):e226. doi: 10.2196/jmir.3429. http://www.jmir.org/2014/9/e226/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Kenwright M, Liness S, Marks I. Reducing demands on clinicians by offering computer-aided self-help for phobia/panic. Feasibility study. Br J Psychiatry. 2001 Nov;179:456–9. doi: 10.1192/bjp.179.5.456. http://bjp.rcpsych.org/cgi/pmidlookup?view=long&pmid=11689405. [DOI] [PubMed] [Google Scholar]
  • 55.Kiluk BD, Devore KA, Buck MB, Nich C, Frankforter TL, LaPaglia DM, Yates BT, Gordon MA, Carroll KM. Randomized trial of computerized cognitive behavioral therapy for alcohol use disorders: efficacy as a virtual stand-alone and treatment add-on compared with standard outpatient treatment. Alcohol Clin Exp Res. 2016 Sep;40(9):1991–2000. doi: 10.1111/acer.13162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Klein AA, Slaymaker VJ, Dugosh KL, McKay JR. Computerized continuing care support for alcohol and drug dependence: a preliminary analysis of usage and outcomes. J Subst Abuse Treat. 2012 Jan;42(1):25–34. doi: 10.1016/j.jsat.2011.07.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Kok RN, van SA, Beekman AT, Cuijpers P. Short-term effectiveness of web-based guided self-help for phobic outpatients: randomized controlled trial. J Med Internet Res. 2014 Sep 29;16(9):e226. doi: 10.2196/jmir.3429. http://www.jmir.org/2014/9/e226/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Kooistra LC, Ruwaard J, Wiersma Je, van Oppen P, van der Vaart R, van Gemert-Pijnen JE, Riper H. Development and initial evaluation of blended cognitive behavioural treatment for major depression in routine specialized mental health care. Internet Interv. 2016 May;4:61–71. doi: 10.1016/j.invent.2016.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Kordy H, Wolf M, Aulich K, Bürgy M, Hegerl U, Hüsing J, Puschner B, Rummel-Kluge C, Vedder H, Backenstrass M. Internet-delivered disease management for recurrent depression: a multicenter randomized controlled trial. Psychother Psychosom. 2016;85(2):91–8. doi: 10.1159/000441951. [DOI] [PubMed] [Google Scholar]
  • 60.Krieger T, Meyer B, Sude K, Urech A, Maercker A, Berger T. Evaluating an e-mental health program (“deprexis”) as adjunctive treatment tool in psychotherapy for depression: design of a pragmatic randomized controlled trial. BMC Psychiatry. 2014 Oct 08;14:285. doi: 10.1186/s12888-014-0285-9. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-014-0285-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Månsson KN, Skagius RE, Gervind E, Dahlin M, Andersson G. Development and initial evaluation of an internet-based support system for face-to-face cognitive behavior therapy: a proof of concept study. J Med Internet Res. 2013 Dec 10;15(12):e280. doi: 10.2196/jmir.3031. http://www.jmir.org/2013/12/e280/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Marks I M, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D. Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med. 2004 Jan;34(1):9–17. doi: 10.1017/s003329170300878x. [DOI] [PubMed] [Google Scholar]
  • 63.Kim SJ, Marsch LA, Acosta MC, Guarino H, Aponte-Melendez Y. Can persons with a history of multiple addiction treatment episodes benefit from technology delivered behavior therapy? A moderating role of treatment history at baseline. Addict Behav. 2016 Mar;54:18–23. doi: 10.1016/j.addbeh.2015.11.009. http://europepmc.org/abstract/MED/26657820. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Nordmo M, Sinding Ai, Carlbring P, Andersson G, Havik Oe, Nordgreen T. Internet-delivered cognitive behavioural therapy with and without an initial face-to-face psychoeducation session for social anxiety disorder: A pilot randomized controlled trial. Internet Interv. 2015 Nov;2(4):429–36. doi: 10.1016/j.invent.2015.10.003. [DOI] [Google Scholar]
  • 65.Pier C, Austin DW, Klein B, Mitchell J, Schattner P, Ciechomski L, Gilson KJ, Pierce D, Shandley K, Wade V. A controlled trial of internet-based cognitive-behavioural therapy for panic disorder with face-to-face support from a general practitioner or email support from a psychologist. Ment Health Fam Med. 2008 Mar;5(1):29–39. http://europepmc.org/abstract/MED/22477844. [PMC free article] [PubMed] [Google Scholar]
  • 66.Shandley K, Austin DW, Klein B, Pier C, Schattner P, Pierce D, Wade V. Therapist-assisted, internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists? J Med Internet Res. 2008 May 19;10(2):e14. doi: 10.2196/jmir.1033. http://www.jmir.org/2008/2/e14/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Reins JA, Ebert DD, Lehr D, Riper H, Cuijpers P, Berking M. Internet-based treatment of major depression for patients on a waiting list for inpatient psychotherapy: protocol for a multi-centre randomised controlled trial. BMC Psychiatry. 2013 Nov 26;13:318. doi: 10.1186/1471-244X-13-318. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/1471-244X-13-318. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Robertson L, Smith M, Castle D, Tannenbaum D. Using the internet to enhance the treatment of depression. Australas Psychiatry. 2006 Dec;14(4):413–7. doi: 10.1080/j.1440-1665.2006.02315.x. [DOI] [PubMed] [Google Scholar]
  • 69.Romijn G, Riper H, Kok R, Donker T, Goorden M, Hakkaart van Roijen L, Kooistra L, van Balkom A, Koning J. Cost-effectiveness of blended vs. face-to-face cognitive behavioural therapy for severe anxiety disorders: study protocol of a randomized controlled trial. BMC Psychiatry. 2015 Dec 12;15:311. doi: 10.1186/s12888-015-0697-1. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-015-0697-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Sethi S, Campbell Aj, Ellis La. The use of computerized self-help packages to treat adolescent depression and anxiety. J Technol Hum Serv. 2010 Aug 31;28(3):144–60. doi: 10.1080/15228835.2010.508317. [DOI] [Google Scholar]
  • 71.Tannenbaum D, Spiranovic C. Severe, recurrent depression managed in a remote setting via the internet: an example of remote care using the healthsteps system. In: Taylor CB, editor. How to Practice Evidence-based Psychiatry: Basic Principles and Case Studies. Arlington, TX: American Psychiatric Publishing, Inc; 2011. pp. 275–86. [Google Scholar]
  • 72.Van Voorhees BW, Ellis JM, Gollan JK, Bell CC, Stuart SS, Fogel J, Corrigan PW, Ford DE. Development and process evaluation of a primary care internet-based intervention to prevent depression in emerging adults. Prim Care Companion J Clin Psychiatry. 2007;9(5):346–55. doi: 10.4088/pcc.v09n0503. http://www.psychiatrist.com/PCC/article/Pages/2007/v09n05/v09n0503.aspx. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Whitfield G, Williams C, Hinshelwood R, Pashley A, Campsie L. The impact of a novel computerized CBT CD-Rom (overcoming depression) offered to patients referred to clinical psychology. Eur Psychiatry. 2007 Mar;22:S248. doi: 10.1016/j.eurpsy.2007.01.831. [DOI] [Google Scholar]
  • 74.Wilhelmsen M, Lillevoll K, Risør MB, Høifødt R, Johansen ML, Waterloo K, Eisemann M, Kolstrup N. Motivation to persist with internet-based cognitive behavioural treatment using blended care: a qualitative study. BMC Psychiatry. 2013 Nov 07;13:296. doi: 10.1186/1471-244X-13-296. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/1471-244X-13-296. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75.Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, Otto MW. Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. Am J Psychiatry. 2005 Jun;162(6):1158–64. doi: 10.1176/appi.ajp.162.6.1158. [DOI] [PubMed] [Google Scholar]
  • 76.Zwerenz R, Becker J, Knickenberg RJ, Hagen K, Dreier M, Wölfling K, Beutel ME. Enhancing inpatient psychotherapeutic treatment with online self-help: study protocol for a randomized controlled trial. Trials. 2015 Mar 17;16:98. doi: 10.1186/s13063-015-0620-6. https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-015-0620-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Carroll KM, Ball SA, Martino S, Nich C, Babuscio TA, Nuro KF, Gordon MA, Portnoy GA, Rounsaville BJ. Computer-assisted delivery of cognitive-behavioral therapy for addiction: a randomized trial of CBT4CBT. Am J Psychiatry. 2008 Jul;165(7):881–8. doi: 10.1176/appi.ajp.2008.07111835. http://europepmc.org/abstract/MED/18450927. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Marsch LA, Guarino H, Acosta M, Aponte-Melendez Y, Cleland C, Grabinski M, Brady R, Edwards J. Web-based behavioral treatment for substance use disorders as a partial replacement of standard methadone maintenance treatment. J Subst Abuse Treat. 2014 Jan;46(1):43–51. doi: 10.1016/j.jsat.2013.08.012. http://europepmc.org/abstract/MED/24060350. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Jacmon J, Malouff J, Taylor N. Treatment of major depression: effectiveness of cognitive behavior therapy with an internet course as a central component. EJAP. 2010 Jan 03;5(2):8. doi: 10.7790/ejap.v5i2.153. [DOI] [Google Scholar]
  • 80.Kenter RM, van SA, Hobbel SH, Smit F, Bosmans J, Beekman A, Cuijpers P. Effectiveness and cost effectiveness of guided online treatment for patients with major depressive disorder on a waiting list for psychotherapy: study protocol of a randomized controlled trial. Trials. 2013 Dec 01;14:412. doi: 10.1186/1745-6215-14-412. https://trialsjournal.biomedcentral.com/articles/10.1186/1745-6215-14-412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81.Murphy SM, Campbell AN, Ghitza UE, Kyle TL, Bailey GL, Nunes EC, Polsky D. Cost-effectiveness of an internet-delivered treatment for substance abuse: Data from a multisite randomized controlled trial. Drug Alcohol Depend. 2016 Apr 01;161:119–26. doi: 10.1016/j.drugalcdep.2016.01.021. http://europepmc.org/abstract/MED/26880594. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Cahill J, Barkham M, Hardy G, Rees A, Shapiro DA, Stiles WB, Macaskill N. Outcomes of patients completing and not completing cognitive therapy for depression. Br J Clin Psychol. 2003 Jun;42(Pt 2):133–43. doi: 10.1348/014466503321903553. [DOI] [PubMed] [Google Scholar]
  • 83.Kenter RM, Cuijpers P, Beekman A, van Straten A. Effectiveness of a web-based guided self-help intervention for outpatients with a depressive disorder: short-term results from a randomized controlled trial. J Med Internet Res. 2016 Mar 31;18(3):e80. doi: 10.2196/jmir.4861. http://www.jmir.org/2016/3/e80/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 84.Kleiboer A, Smit J, Bosmans J, Ruwaard J, Andersson G, Topooco N, Berger T, Krieger T, Botella C, Baños R, Chevreul K, Araya R, Cerga-Pashoja A, Cieślak R, Rogala A, Vis C, Draisma S, van SA, Kemmeren L, Ebert D, Berking M, Funk B, Cuijpers P, Riper H. European COMPARative Effectiveness research on blended Depression treatment versus treatment-as-usual (E-COMPARED): study protocol for a randomized controlled, non-inferiority trial in eight European countries. Trials. 2016 Aug 03;17(1):387. doi: 10.1186/s13063-016-1511-1. https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-016-1511-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 85.Lindhiem O, Bennett CB, Rosen D, Silk J. Mobile technology boosts the effectiveness of psychotherapy and behavioral interventions: a meta-analysis. Behav Modif. 2015 Nov;39(6):785–804. doi: 10.1177/0145445515595198. http://europepmc.org/abstract/MED/26187164. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 86.Rose GL, Skelly JM, Badger GJ, Ferraro TA, Helzer JE. Efficacy of automated telephone continuing care following outpatient therapy for alcohol dependence. Addict Behav. 2015 Feb;41:223–31. doi: 10.1016/j.addbeh.2014.10.022. http://europepmc.org/abstract/MED/25452069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.Moore BA, Fazzino T, Barry DT, Fiellin DA, Cutter CJ, Schottenfeld RS, Ball SA. The recovery line: a pilot trial of automated, telephone-based treatment for continued drug use in methadone maintenance. J Subst Abuse Treat. 2013 Jul;45(1):63–9. doi: 10.1016/j.jsat.2012.12.011. http://europepmc.org/abstract/MED/23375114. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia Appendix 1

Previous psychological research combining Internet and face-to-face psychotherapy.

jmir_v19i9e306_app1.pdf (78.7KB, pdf)
Multimedia Appendix 2

Outcome of previous psychological research combining Internet and face-to-face (f2f) psychotherapy.

jmir_v19i9e306_app2.pdf (87.8KB, pdf)
Multimedia Appendix 3

Study quality assessment of blended treatments.

jmir_v19i9e306_app3.pdf (42.4KB, pdf)

Articles from Journal of Medical Internet Research are provided here courtesy of JMIR Publications Inc.

RESOURCES