Skip to main content
PMC home page
Internet Interventions logoLink to Internet Interventions
. 2018 Jul 26;13:108–115. doi: 10.1016/j.invent.2018.07.006

ICBT in routine care: A descriptive analysis of successful clinics in five countries

Nickolai Titov a,⁎, Blake Dear a, Olav Nielssen b, Lauren Staples a, Heather Hadjistavropoulos c, Marcie Nugent c, Kelly Adlam c, Tine Nordgreen d,f, Kristin Hogstad Bruvik d, Anders Hovland e,f, Arne Repål g, Kim Mathiasen h,i, Martin Kraepelien j, Kerstin Blom d,j, Cecilia Svanborg j, Nils Lindefors j, Viktor Kaldo j,k
PMCID: PMC6112100  PMID: 30206525

Abstract

Clinical trials have demonstrated the efficacy of internet delivered cognitive behaviour therapy (ICBT) for anxiety and depression. However, relatively little is known about the context, operations, and outcomes of ICBT when administered as part of routine care. This paper describes the setting, relationship to existing health services, procedures for referral, assessment, treatment, patients and outcomes of ICBT clinics in Sweden, Denmark, Norway, Canada and Australia.

All five clinics provide services free or at low cost to patients. All have systems of governance to monitor quality of care, patient safety, therapist performance and data security. All five clinics include initial assessments by clinicians and between 10 and 20 min of therapist support during each week. Published reports of outcomes all demonstrate large clinical improvement, low rates of deterioration, and high levels of patient satisfaction. Services that require a face to face assessment treat smaller numbers of patients and have fewer patients from remote locations.

The paper shows that therapist-guided ICBT can be a valuable part of mental health services for anxiety and depression. Important components of successful ICBT services are rigorous governance to maintain a high standard of clinical care, and the measurement and reporting of outcomes.

Keywords: Delivery, Description, Implementation, Internet-delivered cognitive behaviour therapy, Psychological treatment, Routine care, Depression, Anxiety, Disorders

Highlights

  • •

    This study describes internet-delivered cognitive behaviour therapy (ICBT) in routine care in five high income countries

  • •

    Each service uses a comprehensive framework of policies and procedures to ensure patient safety, data safety, and service quality

  • •

    Each clinic measures outcomes to monitor progress and safety, and demonstrate the effectiveness of ICBT in routine care

  • •

    The provision of these services within public health systems enables services to be offered free or at low cost

  • •

    High quality ICBT can be tailored to local contexts and delivered to a large number of patients in routine clinical care

1. Introduction

Despite clear evidence that psychological treatments such as cognitive behavioural therapy (CBT) are effective in treating common mental disorders, less than half of people with anxiety and depression seek treatment in a 12-month period (Wang et al., 2007; Whiteford et al., 2014). Internet delivered Cognitive Behaviour Therapy (ICBT) has been proposed as a way of overcoming some of the barriers to care (Mojtabai et al., 2011; Andersson and Titov, 2014), and there have been numerous clinical trials demonstrating the efficacy of ICBT in the treatment of anxiety and depression (Bee et al., 2008; Cuijpers et al., 2008; Cuijpers et al., 2009; Andrews et al., 2010; Cuijpers et al., 2010; Ruwaard et al., 2011; Hedman et al., 2012; Richards and Richardson, 2012; Andersson and Hedman, 2013; Andersson et al., 2014; Andersson and Titov, 2014; Twomey et al., 2015; Newby et al., 2016; Olthuis et al., 2016; Erbe et al., 2017; Simblett et al., 2017). Importantly, such research has inspired efforts to deploy ICBT as part of routine care, and there have been significant and impressive reports of outcomes from deployment in mental health clinics in several countries (Kaldo-Sandstrom et al., 2004; Hedman et al., 2010; Ruwaard et al., 2012; Andersson and Hedman, 2013; Williams and Andrews, 2013; El Alaoui et al., 2015a; Hadjistavropoulos et al., 2016; Hobbs et al., 2017; Newby et al., 2017; Folker et al., 2018; Nordgreen et al., 2018). However, not all attempts to provide ICBT and other internet-delivered psychological interventions as part of routine care have been successful (Gilbody et al., 2015; Kenter et al., 2015), which raises questions about the effectiveness of ICBT beyond preliminary validating clinical trials (Ruwaard and Kok, 2015; Mohr et al., 2017) and underscores the importance of Phase IV clinical trials which report benefits and risks of interventions when implemented and used in the general population (National Health and Medical Research Council, 2015; National Institutes of Health, 2017). Hence, there is now an urgent need for information about the common features and outcomes of successful clinics to guide effective implementation and delivery of ICBT in routine care.

This paper aims to describe the features of five clinics that have successfully provided ICBT as part of routine care for anxiety and depression; the Internet Psychiatry Clinic (Stockholm, Sweden), the Internetpsykiatrien Clinic (Southern Denmark, Denmark), the eMeistring clinic (Bergen, Norway), the Online Therapy Unit (Regina, Canada), and the MindSpot Clinic (Sydney, Australia). These clinics all provide ICBT interventions already validated in clinical trials, primarily to treat depression and anxiety disorders. Each clinic provides therapist (mental health professional) assessment of symptoms, delivers therapist-guided ICBT rather than offering fully automated treatment, and have all published results of ICBT in routine care in a manner consistent with the aims of Phase IV trials (Hedman et al., 2013; Hedman et al., 2014; El Alaoui et al., 2015a; Nielssen et al., 2015; Titov et al., 2015a; Hadjistavropoulos et al., 2016; Nordgreen et al., 2016; Staples et al., 2016; Hadjistavropoulos et al., 2017; Lee et al., 2017; Titov et al., 2017a; Nordgreen et al., 2018). As the aim of this paper is to describe similarities and differences in context, procedures and patient characteristics, rather than development and implementation, we have intentionally avoided specific frameworks of reporting (Damschroder et al., 2009; Gaglio et al., 2013).

The information presented here draws on contributions to a symposium presented at the 9th Scientific Meeting of the International Society for Research into Internet Interventions (ISRII), held in Berlin, Germany in October 2017 (Titov et al., 2017b). It is acknowledged that these five clinics are not representative of all clinics delivering ICBT, particularly those providing little or no assessment or support (Christensen et al., 2006; Klein et al., 2011; Proudfoot et al., 2013), and that there are an increasing number of similar clinics around the world. Further, we note that the details reported in this paper are presented as descriptive and not prescriptive, as we acknowledge that context and other factors will determine the optimum service model. The data reported here were obtained under ethical approval from each clinics' respective institutional ethics committee or relevant institutional policies, and is current to 31 December 2017.

The first section of the paper provides a brief description of each clinic, followed by a discussion of similarities and differences in 1) context; 2) service models; 3) patient characteristic and 4) outcomes, drawn from published data from each clinic and additional details provided to the first author. We conclude with an attempt to identify factors that contribute to the success of these clinics.

2. Summary of the five clinics

  • 1.

    Sweden

The Internet Psychiatry Clinic (the Swedish clinic) was founded in 2007 in Stockholm and is located at Psychiatry Southwest, at the Stockholm Health Care Services. The clinic operates within specialist psychiatric care but provides all citizens with evidence-based ICBT and the clinic receives recurrent funding as a regular hospital service. Patients pay a small charge for the service at a level consistent with paying for other specialist services. Information about the clinic is available via www.internetpsykiatri.se and patients can learn about the clinic via that site or the national eHealth service www.1177.se. Since beginning operations, the clinic has provided services to roughly 11,000 patients, of whom >5600 have received treatment.

Patients either self-refer or are referred by local doctors or from other psychiatric clinics throughout Sweden. Patients first complete a detailed online questionnaire and then attend a face-to-face psychiatric assessment. Treatment is provided for depression, social anxiety, panic disorder, irritable bowel syndrome and insomnia. People with very severe depression, deemed to be at high risk of suicide, have a language barrier or cognitive impairment, or have complex psychosocial needs are not eligible for ICBT. About half of the assessed patients were not considered suitable for ICBT and were instead referred to more appropriate services. The technical platform for delivering ICBT, named P2, was developed within the clinic and is accessed via the established national eHealth service www.1177.se which provides a secure login.

There is a specific ICBT program for each condition, which have all been evaluated in randomized clinical trials (e.g., Andersson et al., 2005; Andersson et al., 2006; Carlbring et al., 2007; Bergstrom et al., 2010; Ljotsson et al., 2011; Kaldo et al., 2015). Each ICBT program is delivered over 12 weeks except insomnia (9 weeks) and consists most commonly of 10 mostly text-based modules with psychoeducation, exercises and homework reports based on established CBT methods. Therapists are clinical psychologists who interact with patients weekly via secure messages within the platform, SMS messages, or in some cases telephone. Patients receive support from the same therapist throughout treatment. Progress is measured weekly via clinically validated outcome measures which are delivered online, and all patients are asked to complete a more extensive post-treatment online assessment, which if needed is complemented with a telephone call or a regular visit. Patients also receive follow-up with an online assessment at three months post-treatment. The outcomes of each program in routine care are evaluated in quarterly reports of clinic outcomes.

  • 2.

    Denmark

The clinic Internetpsykiatrien (the Danish clinic) began operating in June 2014 as part of the Mental Health Services of Southern Denmark in response to a national action plan for deployment of telemedicine. It was initially funded by the Danish Agency for Digitisation, from 2016 as part routine care by the Region of Southern Denmark and from 2018 it was jointly funded by all five health regions so that clinic services are provided free of charge to patients. The clinic provides guided ICBT to adults with depression and anxiety living in Denmark. Since June 2014, the clinic has provided services to 870 patients, of whom 441 have received treatment.

Patients self-refer via the clinic's web-page (www.internetpsykiatrien.dk) and first complete an online screening questionnaire, followed by a therapist interview by video-conference. People with severe depression or considered to be at high risk of suicide are ineligible for ICBT. As in Sweden, about half of the assessed patients do not start ICBT and are instead referred to more appropriate services. Eligible patients are provided with access to the online treatment programs and are supported weekly by a therapist via either telephone or a secure text module. Therapists are licensed psychologists or psychologists under supervision.

The ICBT programs provided are FearFighter for anxiety disorders (Marks et al., 2004; Mathiasen et al., 2016) and the newly developed program, NoDep for depression (Folker et al., 2018; Mathiasen et al., 2018). Symptoms are measured by self-report questionnaires administered pre- and post-treatment as well as weekly during treatment. Patients receive weekly contact by telephone or secure email by the same therapist.

  • 3.

    Norway

The eMeistring clinic (the Norwegian clinic) began operating in April 2013 at Haukeland University Hospital after three successful ICBT trials in Norway (Nordgreen et al., 2010; Nordmo et al., 2015; Nordgreen et al., 2016) using materials developed at Uppsala University and Linköping University, Sweden. The clinic was initially funded as a project by national, regional and local health authorities. Haukeland University Hospital provides specialist outpatient and inpatient psychiatric care, and added the ICBT clinic as part of routine care. In 2015 two further hospitals in Norway added the eMeistring programs as part of their routine care service.

Information about the clinic is available via the hospital website at www.emeistring.no and Facebook page, and patients can learn about the clinic via their general practitioner (GP), therapist or the clinic website. A total of 1153 patients have received ICBT at the Norwegian eMeistring clinics since April 2013.

The clinic initially only took referrals from GPs. However, from 2018 patients were able to contact the clinic directly for telephone screening, and if they are enrolled in treatment, the GP is informed. The clinics offer guided ICBT to adults (+18) with panic disorder, social anxiety disorder, and mild and moderate depression living in the catchment area of the three hospitals. As in Sweden, patients attend a face-to-face diagnostic assessment to determine suitability and receive information about therapist-guided ICBT as an alternative to face to face treatment. A face-to-face interview is also scheduled after treatment. Patients in need of other forms of treatment are offered this at the hospital.

During treatment patients receive therapist contact once or twice a week via a secure email system. Therapists include psychologists, psychiatrists, social workers and nurses. The clinic offers three ICBT programs, with eight or nine modules delivered over 14 weeks. Symptoms of the primary disorder are measured weekly using a disorder-specific patient reported outcome measure, with other symptoms, outcomes, and satisfaction assessed less frequently. There is a six-month follow-up, and five-year follow up will start at the end of 2018.

  • 4.

    Canada

The Online Therapy Unit (the Canadian clinic) was launched at the University of Regina in Saskatchewan in October 2010. Funding was initially provided by a federal research grant, but since 2015 the Unit has also received funding from the Saskatchewan Ministry of Health. The Unit aims to provide therapist-guided ICBT for depression and anxiety, to educate other mental health providers on delivery of ICBT, and to conduct research on ICBT in routine practice. A collaboration model is used such that both therapists employed by the Unit and those working in publicly funded community clinics deliver ICBT. Services to patients are provided free of charge.

Patients learn about the Unit via providers in community mental health clinics, family physicians, word of mouth, media reports, online searches, and email announcements. These sources direct patients to gain additional information about the clinic by visiting www.onlinetherapyuser.ca. Between October 2010 and December 2017, the clinic provided services to 4209 patients, of whom 2766 received ICBT. Patients first complete an online screening assessment followed by telephone screening assessment. ICBT is offered to adults who endorse at least some symptoms of depression and anxiety, reside in the province of Saskatchewan, are comfortable with using computers, do not require face-to-face care (e.g., high suicide risk, primary problem with alcohol or drugs), and who consent to care and can provide the name of a health care provider to contact in the event of an emergency. Although disorder-specific ICBT programs have been offered, the Unit currently delivers an eight-week five-lesson therapist-guided transdiagnostic ICBT program which targets symptoms of both anxiety and depression, developed at Macquarie University, Australia (Titov et al., 2013; Dear et al., 2015; Titov et al., 2015b). Clinically validated patient reported outcome measures of depression and anxiety are administered with each core lesson, at post-treatment and three-month follow-up.

Treatment is delivered by registered mental health professionals and graduate psychology and social work students under supervision. Patients receive weekly therapist contact via secure email or telephone during the eight weeks of the course, taking an average of 20 min per patient. Therapist contact is intended to: 1) provide support; 2) highlight lesson content; 3) answer patient questions and assist patients in applying skills; and 4) reinforce progress, lesson completion and practice of skills.

  • 5.

    Australia

The MindSpot Clinic (the Australian clinic) was launched in 2013 at Macquarie University, in Sydney, funded by the Australian Government as part of its eMental Health strategy. The aim of the clinic is to improve access to evidence-based education, triage, assessment, referral, and treatment services to adults with symptoms of anxiety and depressive disorders across Australia. Clinic services are provided free to the patient.

Initially most patients learned about the clinic (www.mindspot.org.au) via online advertising and links from other mental health websites, but increasingly referrals have come from other people who have used the clinic. Between January 2013 and December 2017, >80,000 participants registered to use clinic, and >16,000 opted to receive ICBT.

Patients either self-refer (85%) or are referred by a health professional (15%). Patients first register online or via telephone and complete a detailed assessment questionnaire, followed by telephone or secure email contact with a therapist to discuss symptoms and treatment options. Participants then choose between information to assist with self-management, referral to another service or ICBT. The clinic offers seven ICBT programs that have been validated in clinical trials, including transdiagnostic treatments designed to treat symptoms of anxiety and depression (the Wellbeing, Wellbeing Plus, Mood Mechanic, and Indigenous Wellbeing courses), and disorder-specific treatments for obsessive compulsive disorder, post-traumatic stress disorder, and chronic pain (Dear et al., 2013; Titov et al., 2013; Wootton et al., 2013; Johnston et al., 2014; Spence et al., 2014; Wootton et al., 2014; Dear et al., 2015; Titov et al., 2015b; Wootton et al., 2015; Dear et al., 2016; Fogliati et al., 2016; Hadjistavropoulos et al., 2016; Dear et al., 2017). These treatments all comprise five lessons, delivered over eight-weeks. Outcome is measured by course specific symptom scales that are administered online weekly and at three-month post-treatment.

Therapists are all registered or provisionally registered mental health professionals, who contact and monitor participants weekly during treatment via secure email system or by telephone. When preferred, patients can receive a hardcopy of the ICBT programs via post.

3. Key similarities and differences

3.1. Clinic context

This section describes and compares the characteristics of the clinics with respect to context, governance, and funding.

3.1.1. Context

All five clinics operate in high income countries with universal publicly funded healthcare and high internet penetration. All clinics are in jurisdictions where there is support from policy makers and health funding bodies at a national or regional level for the use of technology to increase access to psychological treatments. Further, all clinics operate in jurisdictions in which clinical guidelines recognise CBT as an effective treatment for anxiety and depression. With the exception of the Danish clinic, all clinics were initiatives of publicly funded healthcare services in conjunction with universities, rather than private companies, and maintain relationships with those universities.

With respect to integration into broader health networks, the Swedish and Norwegian clinics are located directly within existing mental health services. The Danish clinic was initially located within a mental health research and development centre but is now available to all health regions. The Canadian clinic was developed within a university but now is located both in the university and community clinics. The Australian clinic was developed and is operated by a university and is not located within traditional healthcare services. Despite these differences, all five clinics encourage patients to engage with a primary care physician, and all provide clinical reports to health professionals involved in a patient's mental healthcare.

3.1.2. Governance, quality assurance and evaluation

Each clinic operates according to a framework of policies and procedures that are consistent with relevant national standards for the provision of mental health services. These describe conduct of clinical activities, including quality of care, patient safety, supervision of therapists, routine measurement of outcomes, and assessment and treatment procedures (see below). Each clinic also operates IT security frameworks that are consistent with national standards, and which prescribe the safe and responsible use of IT systems in delivering clinical care.

3.1.3. Funding

All five clinics obtain public funding but, reflecting local contexts, each clinic is funded in a slightly different way. The three European-based clinics started as specifically funded projects, but now receive recurrent funding via the hospital or public health service with which they are associated, equivalent to the funding of traditional care within those services. Patients of the Swedish clinics pay a small charge to use the service, which is consistent with that charged by other specialist health services, and the reimbursement system in both Sweden and Norway now includes ICBT. The Canadian and Australian clinics also receive recurrent funding, but as project grants, rather than as part of the usual care provided by the public health system.

3.2. Clinic services

This section first describes the number of patients served by each clinic and details about referrals. This is followed by a description of target conditions, eligibility and exclusion criteria, details of assessment, treatment, therapists, outcome measures, operating hours, and management of patient safety.

3.2.1. Geographic catchment and referrals

The Swedish, Norwegian, and Australian clinics, and since 2018, the Danish clinic, provide services nationally. The Canadian clinic services the vast sparsely populated province of Saskatchewan. Thus, the catchment of possible users varies from approximately 1.1 million in Saskatchewan, to 24.1 million in Australia (based on 2016 population estimates).

The clinics vary considerably with respect to volume, ranging from 280 people assessed in 2017 by the Danish clinic, to 20,425 assessed by the Australian clinic during the same period. These differences reflect differences in geographic catchment areas, funding arrangements, number of therapists, the age of each clinic and hence the awareness of the clinic in the community and the maturity of referral pathways, the types of services provided and the assessment methods (see below).

All clinics accept self-referrals, and with the exception of the Danish clinic, also accept referrals from health professionals. Until 2018 the Norwegian clinic only accepted patients referred by a health professional, and still requires patients to nominate a treating doctor.

3.2.2. Eligibility and assessment

All five clinics primarily treat anxiety and depressive disorders. The Swedish clinic also provides treatment courses for irritable bowel syndrome and insomnia, and the Australian and Canadian clinics offer a treatment course for chronic pain. These differences show how the service platform and core features of ICBT can be adapted to successfully treat a range of chronic psychological and physical health conditions.

All five clinics require patients to complete an online questionnaire providing demographic details, history of symptoms, treatment and current symptoms, using validated clinical measures of target symptoms. The Australian clinic also permits patients to complete this process via telephone when they do not have access to the internet.

After completing the online questionnaires, all patients are assessed by a therapist, which at, the three clinics in Europe involves an interview in-person or by videoconference, and at the Canadian and Australian clinics via telephone or in some circumstances, via a secure text-based messaging system. In Sweden there were attempts to delegate the assessment to other psychiatric clinics or primary care, but they have not been pursued due to problems with the quality of assessment, and the small numbers assessed.

In all clinics the online questionnaires act as an indirect method of identifying patients with cognitive impairment, low motivation, or those unable to use the necessary technology. The second assessment step allows the therapist to confirm the findings of the online assessment and to determine suitability of treatment, including ensuring the patient's understanding of the demands of ICBT treatment. It should be noted that about half the patients at the clinics in Sweden and Norway are referred to other services at the time of the assessment, and that two thirds of those completing assessments at the Australian clinic report that they are seeking assessment and information, rather than ICBT treatment, which allowed that clinic to provide these additional services to a very large number of people.

All five clinics provide services to adults, with the Danish, Norwegian, Canadian, and Australian clinics having a minimum age of 18, and Swedish clinic providing services to people from the age of 16 years. The three clinics in Europe require users to meet diagnostic criteria for the target disorder. This is not a specific requirement for the Canadian and Australian clinics providing the patients are assessed as likely to benefit from ICBT. Exclusion criteria across clinics are similar, with all clinics excluding patients who indicate an imminent risk of suicide, have untreated psychosis, or who are considered unlikely to benefit from this model of treatment because of other problems (e.g. severe substance use disorders). Excluded patients are referred to an appropriate service for further care.

3.2.3. Treatment, processes, and therapists

Consistent with differences in assessment numbers, the clinics vary considerably with respect to volume of patients treated, ranging from 183 people treated by the Danish clinic in 2017, to 4,054 treated by the Australian clinic during the same period. The differences reflect the age of each clinic, as the number of patients has tended to increase as the service becomes known, and also on therapist time required for each assessment.

All five clinics deliver ICBT interventions previously validated in clinical trials, and which aim to provide CBT content equivalent to that provided during traditional face to face CBT. The content of each of these interventions comprise text, images and graphics, delivered as a series of modules systematically released during treatment. None of the clinics use multimedia or apps for mobile devices, although all content is accessible via mobile devices. The Australian clinic makes treatment material available by post because of the limited internet reliability and access in parts of the country and also because of patient preference. The duration of treatment varies from eight to fourteen weeks across clinics, with follow-up outcome measures administered at three or six months post-treatment.

The treatment procedures are similar across clinics, with patients encouraged to log in regularly to read materials, practice skills, and complete outcome measures. Most clinics use a combination of automated and individualised messages to increase adherence and engagement. All clinics also contact patients weekly to monitor progress and support therapy either via a secure messaging system that requires logging into the patient management system, or telephone, with therapists typically spending 10–20 minute duration per week with each patient. Some differences exist with respect to progression of treatment, with the Swedish and Norwegian clinics requiring that patients submit a report of their homework that should be deemed by the therapist to be satisfactory, before the patient can continue. In the Norwegian clinic the therapist reviews the activity and symptoms before deciding whether to provide access to the next module. The Canadian and Australian clinics do not require a homework report, but instead require patients to complete a prior lesson and a weekly symptom questionnaire before they are able to access subsequent lessons.

The therapists in each of the five clinics are either registered or provisionally registered mental health professionals. The Canadian and Swedish clinics also provide supervised training for graduate students, which the Australian clinic will introduce in 2018. The Danish and Swedish clinics only employ psychologists and psychiatrists as therapists, whereas the Canadian, Norwegian and Australian clinics also employ social workers, nurses and counsellors with tertiary qualifications. All clinics provide initial training in systems of care, and regular clinical supervision. Each of the clinics employs both full-time and part-time therapists, with some of the latter also involved in face-to-face care elsewhere.

Three of the clinics operate entirely from one location. The Norwegian clinic operates from several locations whilst the Canadian clinic allows mental health clinicians working in community mental health clinics across the province of Saskatchewan to deliver ICBT, providing training and governance requirements are met.

3.2.4. Outcome measurement

The five clinics all employ validated and widely used patient reported outcome measures to monitor individual progress in treatment and for service evaluation. The measures include symptom scales for depression and anxiety, relevant disorder-specific measures, and measures of patient satisfaction. Several of the clinics also administer measures of disability. All clinics administer symptom measures at assessment, regularly during treatment, and at follow-up. All the clinics regularly review outcomes of routine care, including estimates of clinical improvement and deterioration, and patient satisfaction.

3.2.5. Operating hours

All clinics have online screening and treatment materials available at all hours. Four of the clinics have therapists available to speak with patients during the same hours as the local mental health services. In order to be available to patients in the three time zones across the country, the Australian clinic rosters therapists for extended hours, six days per week.

3.2.6. Management of patient safety

All clinics have carefully developed procedures for identifying, monitoring, and managing patients who might be at risk, both for various harms and suicide, and for those patients who have high symptoms scores or are not improving during treatment (Nielssen et al., 2015). Because of the nature of the treated conditions, it is common for patients to disclose thoughts of suicide and even suicidal plans. Each of the clinics accepts patients who have disclosed suicidal ideation, provided they agree to cooperate with treatment and adhere to a safety plan.

3.3. Patient characteristics, and outcomes

This section describes the characteristics of patients and clinic outcomes. Note that not all clinics collect the same demographic information and that differences reflect multiple factors, including differences between clinics in referrals and recruitment, patient age, and patterns of help-seeking.

3.3.1. Patient characteristics

The mean age of patients using the clinics ranged from 33 to 39 years, with a wide age range, reflecting use by older adults. All five clinics have a larger proportion of female users, broadly consistent with the population prevalence estimates of depression and anxiety disorders in each country and higher treatment seeking by women. The proportion of users across clinics with university level education ranges from 34% to 51%, and the proportion employed ranges from 53% to 81%. A higher proportion of users of the Canadian and Australian clinics live outside cities, reflecting the ability of those clinics to provide assessments remotely via telephone or online.

Data indicate that 25%, 37%, 58%, and 62% are taking prescription psychotropic medications in the Danish, Australian, Canadian and Norwegian clinics respectively. The proportion who reported never having used mental health services is 23%, 26%, and 38% in the Norwegian, Canadian, and Australian clinics, respectively.

An important finding is the severity of symptoms at assessment, with all five clinics reporting mean anxiety and depression symptom scores in the moderate to severe ranges. This is in contrast to the expectation that ICBT would be used to treat people with less severe disorders, and also often in contrast to the clinical trials which in some cases excluded patients with high scores.

3.3.2. Treatment outcomes

All clinics have published outcome studies conducted in routine care (Nordgreen et al., n.d.; Hedman et al., 2013; Hedman et al., 2014; El Alaoui et al., 2015b; Nielssen et al., 2015; Titov et al., 2015a; Hadjistavropoulos et al., 2016; Mathiasen et al., 2016; Nordgreen et al., 2016; Staples et al., 2016; Hadjistavropoulos et al., 2017; Titov et al., 2017a; Folker et al., 2018; Nordgreen et al., 2018; Mathiasen et al., 2018). These results indicate that a large proportion of patients in each of the clinics were rated as treatment completers, that is, they received a minimum therapeutic dose, defined as completing a minimum number of the total number of treatment modules.

All of the clinics reported large within group effect sizes (Cohen's d ≥ 0.8) on primary outcome measures, with a significant proportion of users meeting criteria for clinically significant change. Outcomes studies also reported moderate to large effect sizes (Cohen's d ≥ 0.5) on secondary outcome measures, and improvements in both primary and secondary measures were sustained at three or six-month follow-up. Importantly, the proportion of all users who experience clinically significant deterioration is low, and satisfaction rates across all clinics is high. Further details are available in the individual published results of each clinic.

4. Discussion

This paper summarises the key features of clinics in five countries that have successfully provided therapist-guided ICBT for adults with anxiety and depression as part of routine care. The interventions were deemed to be successful because they have provided effective treatment to large numbers of patients, and have progressed from pilot or project models to permanent services with sustainable funding.

With regards to contextual similarities, all five clinics are located in high income countries with publicly funded healthcare and high internet use. Although ICBT is a comparatively low cost treatment, each of the services received adequate funding to establish sound models of clinical and IT governance at the outset comprising clearly defined clinical protocols, policies and procedures aligned with the relevant national mental health service standards, as well as documented IT governance procedures. All five clinics are specialised health care units providing services to a defined population, although the Canadian model includes therapists operating in community mental health services across the province who have been trained by staff from the centralized unit. All five clinics employ trained therapists who conduct assessments and deliver evidence-based treatment. Further, all five clinics use validated symptom scales to measure progress in treatment and outcomes, have well developed procedures for managing patient safety and have published their outcomes. Adhering to clinic procedures in order to provide high quality care to each patient, and monitoring and responding to patient symptom scores and reported distress have been identified as the main strategies for reducing some of the risks associated with providing treatment remotely (Nielssen et al., 2015; Ruwaard and Kok, 2015).

All of the clinics report encouraging clinical outcomes from ICBT, with sustained and clinically significant improvements in both primary and secondary outcome measures, low deterioration rates and high user satisfaction. This finding is in contrast to reports of less promising outcomes of ICBT provided as part of routine care elsewhere (Gilbody et al., 2015; Kivi et al., 2014).

There are some important differences in the modes of assessment and the types of treatment offered, reflecting funding models and the health systems of the countries in which the clinics operate, geographical challenges, as well as local needs and expectations. For example, three of the clinics primarily provide treatment, whereas an important function of both the Swedish and particularly the Australian clinic is to provide assessments and referral advice. There are also differences in the ICBT programs and the conditions for which treatment is offered. These differences highlight the potential to provide remote and partly automated treatment for a broad range of mental health conditions, as well as triage, diagnostic assessment, education about various disorders and referrals to other services.

There are also important differences in who uses the clinics, reflecting differences in the source of referrals. For example, the clinics with the highest proportion of users who are also taking psychotropic medications are the clinics who receive most, if not all, referrals from health professionals. Because of their geography, location and mode of referral and assessment, >30% of people served by the Australian and Canadian clinics live outside metropolitan areas or in geographically remote locations. This confirms the potential of ICBT to overcome the barrier of distance to care, as well as overcoming some of the barriers to care experienced by people living in metropolitan areas.

4.1. Key success factors

There were eight factors that all five clinics identified as contributing to their successful results, adding to the growing body of published research on the factors contributing to the successful delivery of ICBT in routine care (e.g., Hedman et al., 2012; Ruwaard et al., 2012; Andersson and Hedman, 2013; Erbe et al., 2017), and psychological services in general (e.g., Richards et al., 2016).

First, we note that the clinics have strong clinical, IT, and organisational governance. All clinics recognise that, whilst emerging from academic environments, they need to meet the standards and levels of accountability expected of contemporary mental health services. All the clinics have developed effective procedures for monitoring progress in treatment and patient safety.

Second, all clinics have strong links with funding bodies, local health services and universities. Among other advantages, this has ensured services are developed in an integrated way, adding value to and avoiding duplication of existing services, and facilitating the acceptance of this model of care. The links to research units have provided opportunities for the development and deployment of new treatments, including treatments for a broader range of conditions. In some cases, the links to universities have provided the opportunity to participate in the training of a new generation of therapists.

Third, all five clinics comprise specialised and centralized health care units which provide services across the target area. We propose that the successful delivery of this model of care can be difficult to achieve, at least initially, without established expertise and concentration of effort to establish clinic procedures and train therapists in the model of care, and the lack of expertise or attention to those details may results in poor outcomes. The Canadian clinic is a blended model with both a central unit and local therapists across Saskatchewan, but the local therapists are all trained in providing ICBT by staff from the central unit.

Fourth, all clinics use ICBT programs that have been shown to be effective in clinical trials before their deployment in routine care. The procedures of clinics administering those programs in routine care include systems for maximising fidelity to the proven treatments and minimising therapist drift. All services also have the option of matching the treatments offered or recommended to the clinical needs, expectations and preferences of the patients, including the referral to other services when indicated.

Fifth, all clinics monitor patient satisfaction and use patient advice and feedback to improve clinic procedures, just as patient feedback during clinical trials was important in developing treatment programs. Responding to the views of consumers is consistent with expectations of contemporary clinical services.

Sixth, all clinics have well developed systems to monitor quality care, in particular the regular monitoring of progress in treatment. Monitoring progress and outcome is a hallmark of large scale initiatives to treat high prevalence disorders in other countries (Richards and Borglin, 2011; Clark et al., 2018). Other measures to maintain quality of care include systems of supervision of therapists, ongoing training, and systems for monitoring and management of patients deemed to be at risk. Moreover, all five clinics have published, or are in the process of publishing, their results in peer reviewed journals.

Seventh, all clinics accept self-referrals as well as referrals from health professionals, which we believe eliminates a significant barrier to care, and can also have the effect of attracting better informed and more motivated patients. Accepting self-referrals also increases the number of people who are able to access ICBT and allows the clinics to operate on a greater scale.

Finally, we note that each clinic has developed efficient processes for conducting online and telephone assessments, and in delivering therapist-guided ICBT to large numbers of patients, including systems for training and supervising therapists to manage large volumes of referrals (Hadjistavropoulos et al., 2018) We believe thorough assessment and preparation for treatment, and training and supervision of the delivery workforce, are key features of the good results reported by these clinics. The lack of proper assessment and preparation for treatment can contribute to misunderstanding of expectations in treatment, reduced motivation and engagement, which in turn might explain poorer outcomes reported elsewhere (e.g., Gilbody et al., 2015).

4.2. Strengths and limitations

The main strength of this paper is that it integrates information from clinics in five countries located across three continents, each of which has published results from providing ICBT as part of routine care. Such information is essential for identifying the longer-term benefits and risks of this model of care in the general population, consistent with the intention of Phase IV clinical trials (National Health and Medical Research Council, 2015; National Institutes of Health, 2017). An obvious limitation is that this paper does not include all the providers of ICBT in routine care, of which there are an increasing number of impressive examples and models, and hence the findings might not be representative of the experiences of other clinics. An additional limitation is that the information presented here is necessarily brief, and we refer interested readers to the relevant published literature about each clinic. Furthermore, the clinics reported here provide therapist-guided services, and thus the results may not extend to units delivering self-guided ICBT. We note that there are a growing number of services delivering ICBT or other online psychological therapies, and emerging blended models of care (https://www.e-compared.eu/), and emphasise that the findings in this paper should be considered descriptive and not prescriptive, as the context and other factors will determine the optimum service model. Finally, we have not discussed the challenges in operating ICBT (e.g., costs, web application, retaining personnel) or future directions (Folker et al., 2018). These latter topics will be the focus of another publication.

4.3. Conclusions

This paper describes characteristics of five clinics in five countries successfully delivering therapist-guided ICBT in routine care. The success of these clinics reflects their inception as specialised ICBT services, the alignment with other mental health services and universities, strong clinical, IT and organisational governance, use of well-developed and validated ICBT programs, the safe and effective provision of ICBT programs implemented by well trained and supported therapists, engagement with consumers, and commitment to transparent reporting of outcomes.

Finally, we note that consistent with models of quality assurance and improvement, the service models of all five clinics continue to evolve and change. However, we hope the brief descriptions included here assist other clinics develop ICBT services which are safe, effective, and valued by patients.

Declaration and funding

Dr. Titov and Dr. Dear are authors and developers of the treatment courses used at the MindSpot Clinic but derive no personal or financial benefit from them. Dr. Titov and Dr. Dear are funded by the Australian Government to develop and provide a free national online and telephone-delivered treatment service. Dr. Nielssen is a member of a Lundbeck advisory board for an antipsychotic medication. The other authors report no financial relationships with commercial interests.

This study was investigator initiated. It was funded by Departmental funds from each author's respective university.

Acknowledgements

Each of the clinics gratefully acknowledge their funders, supporting institutions, the patients for allowing the use of their data, and their clinical, management, and technical teams for their efforts in launching and operating the Clinics. We also acknowledge the achievements and leadership shown by early and current researchers in the field of psychological internet interventions. Their work has strongly influenced the models and outcomes reported here and continue to inspire our attempts at reducing barriers to evidence-based psychological services.

References

  1. Andersson G., Hedman E. Effectiveness of guided Internet-delivered cognitive behaviour therapy in regular clinical settings. Verhaltenstherapie. 2013;23:140–148. [Google Scholar]
  2. Andersson G., Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry. 2014;13(1):4–11. doi: 10.1002/wps.20083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Andersson G., Bergstrom J., Hollandare F., Carlbring P., Kaldo V., Ekselius L. Internet-based self-help for depression: randomised controlled trial. Br. J. Psychiatry. 2005;187:456–461. doi: 10.1192/bjp.187.5.456. [DOI] [PubMed] [Google Scholar]
  4. Andersson G., Carlbring P., Holmstrom A., Sparthan E., Furmark T., Nilsson-Ihrfelt E., Buhrman M., Ekselius L. Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial. J. Consult. Clin. Psychol. 2006;74(4):677–686. doi: 10.1037/0022-006X.74.4.677. [DOI] [PubMed] [Google Scholar]
  5. Andersson G., Cuijpers P., Carlbring P., Riper H., Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. 2014;13(3):288–295. doi: 10.1002/wps.20151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Andrews G., Cuijpers P., Craske M.G., McEvoy P., Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010;5(10) doi: 10.1371/journal.pone.0013196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bee P.E., Bower P., Lovell K., Gilbody S., Richards D., Gask L., Roach P. Psychotherapy mediated by remote communication technologies: a meta-analytic review. BMC Psychiatry. 2008;8:60. doi: 10.1186/1471-244X-8-60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bergstrom J., Andersson G., Ljotsson B., Ruck C., Andreewitch S., Karlsson A., Carlbring P., Andersson E., Lindefors N. Internet- versus group-administered cognitive behaviour therapy for panic disorder in a psychiatric setting: a randomised trial. BMC Psychiatry. 2010;10:54. doi: 10.1186/1471-244X-10-54. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Carlbring P., Gunnarsdottir M., Hedensjo L., Andersson G., Ekselius L., Furmark T. Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. Br. J. Psychiatry. 2007;190:123–128. doi: 10.1192/bjp.bp.105.020107. [DOI] [PubMed] [Google Scholar]
  10. Christensen H., Griffiths K.M., Mackinnon A.J., Brittliffe K. Online randomized controlled trial of brief and full cognitive behaviour therapy for depression. Psychol. Med. 2006;36(12):1737–1746. doi: 10.1017/S0033291706008695. [DOI] [PubMed] [Google Scholar]
  11. Clark D.M., Canvin L., Green J., Layard R., Pilling S., Janecka M. Transparency about the outcomes of mental health services (IAPT approach): an analysis of public data. Lancet. 2018;391(10121):679–686. doi: 10.1016/S0140-6736(17)32133-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cuijpers P., van Straten A., Andersson G., van Oppen P. Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. J. Consult. Clin. Psychol. 2008;76(6):909–922. doi: 10.1037/a0013075. [DOI] [PubMed] [Google Scholar]
  13. Cuijpers P., Marks I.M., van Straten A., Cavanagh K., Gega L., Andersson G. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn. Behav. Ther. 2009;38(2):66–82. doi: 10.1080/16506070802694776. [DOI] [PubMed] [Google Scholar]
  14. Cuijpers P., Donker T., van Straten A., Li J., Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol. Med. 2010;40(12):1943–1957. doi: 10.1017/S0033291710000772. [DOI] [PubMed] [Google Scholar]
  15. Damschroder L.J., Aron D.C., Keith R.E., Kirsh S.R., Alexander J.A., Lowery J.C. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement. Sci. 2009;4:50. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Dear B.F., Titov N., Perry K.N., Johnston L., Wootton B.M., Terides M.D., Rapee R.M., Hudson J.L. The pain course: a randomised controlled trial of a clinician-guided internet-delivered cognitive behaviour therapy program for managing chronic pain and emotional well-being. Pain. 2013;154(6):942–950. doi: 10.1016/j.pain.2013.03.005. [DOI] [PubMed] [Google Scholar]
  17. Dear B.F., Staples L.G., Terides M.D., Karin E., Zou J., Johnston L., Gandy M., Fogliati V.J., Wootton B.M., McEvoy P.M., Titov N. Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for generalized anxiety disorder and comorbid disorders: a randomized controlled trial. J. Anxiety Disord. 2015;36:63–77. doi: 10.1016/j.janxdis.2015.09.003. [DOI] [PubMed] [Google Scholar]
  18. Dear B.F., Staples L.G., Terides M.D., Fogliati V.J., Sheehan J., Johnston L., Kayrouz R., Dear R., McEvoy P.M., Titov N. Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for Social Anxiety Disorder and comorbid disorders: a randomized controlled trial. J. Anxiety Disord. 2016;42:30–44. doi: 10.1016/j.janxdis.2016.05.004. [DOI] [PubMed] [Google Scholar]
  19. Dear B.F., Fogliati V.J., Fogliati R., Johnson B., Boyle O., Karin E., Gandy M., Kayrouz R., Staples L.G., Titov N. Treating anxiety and depression in young adults: a randomised controlled trial comparing clinician-guided versus self-guided Internet-delivered cognitive behavioural therapy. Aust. N. Z. J. Psychiatry. 2017;52(7):668–679. doi: 10.1177/0004867417738055. [DOI] [PubMed] [Google Scholar]
  20. El Alaoui S., Hedman E., Kaldo V., Hesser H., Kraepelien M., Andersson E., Ruck C., Andersson G., Ljotsson B., Lindefors N. Effectiveness of Internet-based cognitive-behavior therapy for social anxiety disorder in clinical psychiatry. J. Consult. Clin. Psychol. 2015;83(5):902–914. doi: 10.1037/a0039198. [DOI] [PubMed] [Google Scholar]
  21. El Alaoui S., Hedman E., Ljotsson B., Lindefors N. Long-term effectiveness and outcome predictors of therapist-guided internet-based cognitive-behavioural therapy for social anxiety disorder in routine psychiatric care. BMJ Open. 2015;5(6) doi: 10.1136/bmjopen-2015-007902. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Erbe D., Eichert H.C., Riper H., Ebert D.D. Blending face-to-face and internet-based interventions for the treatment of mental disorders in adults: systematic review. J. Med. Internet Res. 2017;19(9) doi: 10.2196/jmir.6588. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Fogliati V.J., Terides M.D., Gandy M., Staples L.G., Johnston L., Karin E., Rapee R.M., Titov N., Dear B.F. Psychometric properties of the Mini-Social Phobia Inventory (Mini-SPIN) in a large online treatment-seeking sample. Cogn. Behav. Ther. 2016;45(3):236–257. doi: 10.1080/16506073.2016.1158206. [DOI] [PubMed] [Google Scholar]
  24. Folker A.P., Mathiasen K., Lauridsen S.M., Stenderup E., Dozeman E., Folker M.P. Implementing internet-delivered cognitive behavior therapy for common mental health disorders: a comparative case study of implementation challenges perceived by therapists and managers in five European internet services. Internet Interv. 2018;11:60–70. doi: 10.1016/j.invent.2018.02.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Gaglio B., Shoup J.A., Glasgow R.E. The RE-AIM framework: a systematic review of use over time. Am. J. Public Health. 2013;103(6):e38–e46. doi: 10.2105/AJPH.2013.301299. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Gilbody S., Littlewood E., Hewitt C., Brierley G., Tharmanathan P., Araya R., Barkham M., Bower P., Cooper C., Gask L., Kessler D., Lester H., Lovell K., Parry G., Richards D.A., Andersen P., Brabyn S., Knowles S., Shepherd C., Tallon D., White D. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ. 2015;351:h5627. doi: 10.1136/bmj.h5627. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hadjistavropoulos H.D., Nugent M.M., Alberts N.M., Staples L., Dear B.F., Titov N. Transdiagnostic Internet-delivered cognitive behaviour therapy in Canada: an open trial comparing results of a specialized online clinic and nonspecialized community clinics. J. Anxiety Disord. 2016;42:19–29. doi: 10.1016/j.janxdis.2016.05.006. [DOI] [PubMed] [Google Scholar]
  28. Hadjistavropoulos H.D., Nugent M.M., Dirkse D., Pugh N. Implementation of internet-delivered cognitive behavior therapy within community mental health clinics: a process evaluation using the consolidated framework for implementation research. BMC Psychiatry. 2017;17(1):331. doi: 10.1186/s12888-017-1496-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Hadjistavropoulos H.D., Schneider L.H., Klassen K., Dear B.F., Titov N. Development and evaluation of a scale assessing therapist fidelity to guidelines for delivering therapist-assisted Internet-delivered cognitive behaviour therapy. Cogn. Behav. Ther. 2018:1–15. doi: 10.1080/16506073.2018.1457079. [DOI] [PubMed] [Google Scholar]
  30. Hedman E., Ljótsson B., Andersson E., Rück C., Andersson G., Lindefors N. Effectiveness and cost offset analysis of group CBT for hypochondriasis delivered in a psychiatric setting: an open trial. Cogn. Behav. Ther. 2010;39(4):239–250. doi: 10.1080/16506073.2010.496460. [DOI] [PubMed] [Google Scholar]
  31. Hedman E., Ljotsson B., Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev. Pharmacoecon. Outcomes Res. 2012;12(6):745–764. doi: 10.1586/erp.12.67. [DOI] [PubMed] [Google Scholar]
  32. Hedman E., Ljotsson B., Ruck C., Bergstrom J., Andersson G., Kaldo V., Jansson L., Andersson E., Andersson E., Blom K., El Alaoui S., Falk L., Ivarsson J., Nasri B., Rydh S., Lindefors N. Effectiveness of internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatr. Scand. 2013;128(6):457–467. doi: 10.1111/acps.12079. [DOI] [PubMed] [Google Scholar]
  33. Hedman E., Ljotsson B., Kaldo V., Hesser H., El Alaoui S., Kraepelien M., Andersson E., Ruck C., Svanborg C., Andersson G., Lindefors N. Effectiveness of Internet-based cognitive behaviour therapy for depression in routine psychiatric care. J. Affect. Disord. 2014;155:49–58. doi: 10.1016/j.jad.2013.10.023. [DOI] [PubMed] [Google Scholar]
  34. Hobbs M.J., Mahoney A.E.J., Andrews G. Integrating iCBT for generalized anxiety disorder into routine clinical care: treatment effects across the adult lifespan. J. Anxiety Disord. 2017;51:47–54. doi: 10.1016/j.janxdis.2017.09.003. [DOI] [PubMed] [Google Scholar]
  35. Johnston L., Dear B.F., Gandy M., Fogliati V.J., Kayrouz R., Sheehan J., Rapee R.M., Titov N. Exploring the efficacy and acceptability of Internet-delivered cognitive behavioural therapy for young adults with anxiety and depression: an open trial. Aust. N. Z. J. Psychiatry. 2014;48(9):819–827. doi: 10.1177/0004867414527524. [DOI] [PubMed] [Google Scholar]
  36. Kaldo V., Jernelov S., Blom K., Ljotsson B., Brodin M., Jorgensen M., Kraepelien M., Ruck C., Lindefors N. Guided internet cognitive behavioral therapy for insomnia compared to a control treatment - a randomized trial. Behav. Res. Ther. 2015;71:90–100. doi: 10.1016/j.brat.2015.06.001. [DOI] [PubMed] [Google Scholar]
  37. Kaldo-Sandstrom V., Larsen H.C., Andersson G. Internet-based cognitive-behavioral self-help treatment of tinnitus: clinical effectiveness and predictors of outcome. Am. J. Audiol. 2004;13(2):185–192. doi: 10.1044/1059-0889(2004/023). [DOI] [PubMed] [Google Scholar]
  38. Kenter R.M.F., van de Ven P.M., Cuijpers P., Koole G., Niamat S., Gerrits R.S., Willems M., van Straten A. Costs and effects of Internet cognitive behavioral treatment blended with face-to-face treatment: results from a naturalistic study. Internet Interv. 2015;2(1):77–83. [Google Scholar]
  39. Kivi M., Eriksson M.C.M., Hange D., Petersson E.-L., Vernmark K., Johansson B., Björkelund C. Internet-based therapy for mild to moderate depression in Swedish primary care: short term results from the PRIM-NET randomized controlled trial. Cogn. Behav. Ther. 2014;43(4):289–298. doi: 10.1080/16506073.2014.921834. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Klein B., Meyer D., Austin D.W., Kyrios M. Anxiety online: a virtual clinic: preliminary outcomes following completion of five fully automated treatment programs for anxiety disorders and symptoms. J. Med. Internet Res. 2011;13(4) doi: 10.2196/jmir.1918. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Lee Y.C., Gao L., Dear B.F., Titov N., Mihalopoulos C. The cost-effectiveness of the online MindSpot Clinic for the treatment of depression and anxiety in Australia. J. Ment. Health Policy Econ. 2017;20(4):155–166. [PubMed] [Google Scholar]
  42. Ljotsson B., Hedman E., Andersson E., Hesser H., Lindfors P., Hursti T., Rydh S., Ruck C., Lindefors N., Andersson G. Internet-delivered exposure-based treatment vs. stress management for irritable bowel syndrome: a randomized trial. Am. J. Gastroenterol. 2011;106(8):1481–1491. doi: 10.1038/ajg.2011.139. [DOI] [PubMed] [Google Scholar]
  43. Marks I.M., Kenwright M., McDonough M., Whittaker M., Mataix-Cols D. Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol. Med. 2004;34(1):9–17. doi: 10.1017/s003329170300878x. [DOI] [PubMed] [Google Scholar]
  44. Mathiasen K., Riper H., Ehlers L.H., Valentin J.B., Rosenberg N.K. Internet-based CBT for social phobia and panic disorder in a specialised anxiety clinic in routine care: results of a pilot randomised controlled trial. Internet Interv. 2016;4:92–98. doi: 10.1016/j.invent.2016.03.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Mathiasen K., Riper H., Andersen T.E., Roessler K.K. 2018. Evaluation of Clinical Effect in Routine Secondary Care of Guided Internet Based CBT for Adult Depression and Anxiety: An Observational Study. JMIR Preprints. 01/05/2018:10927. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Mohr D.C., Weingardt K.R., Reddy M., Schueller S.M. Three problems with current digital mental health research … and three things we can do about them. Psychiatr. Serv. 2017;68(5):427–429. doi: 10.1176/appi.ps.201600541. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Mojtabai R., Olfson M., Sampson N.A., Jin R., Druss B., Wang P.S., Wells K.B., Pincus H.A., Kessler R.C. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol. Med. 2011;41(8):1751–1761. doi: 10.1017/S0033291710002291. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. National Health and Medical Research Council Phases of clinical trials. 2015. https://www.australianclinicaltrials.gov.au/what-clinical-trial/phases-clinical-trials Retrieved 24.07.2018, from.
  49. National Institutes of Health NIH clinical research trials and you. 2017. https://www.nih.gov/health-information/nih-clinical-research-trials-you/basics Retrieved 24.07.2018, from.
  50. Newby J.M., Twomey C., Yuan Li S.S., Andrews G. Transdiagnostic computerised cognitive behavioural therapy for depression and anxiety: a systematic review and meta-analysis. J. Affect. Disord. 2016;199:30–41. doi: 10.1016/j.jad.2016.03.018. [DOI] [PubMed] [Google Scholar]
  51. Newby J.M., Mewton L., Andrews G. Transdiagnostic versus disorder-specific internet-delivered cognitive behaviour therapy for anxiety and depression in primary care. J. Anxiety Disord. 2017;46:25–34. doi: 10.1016/j.janxdis.2016.06.002. [DOI] [PubMed] [Google Scholar]
  52. Nielssen O., Dear B.F., Staples L.G., Dear R., Ryan K., Purtell C., Titov N. Procedures for risk management and a review of crisis referrals from the MindSpot Clinic, a national service for the remote assessment and treatment of anxiety and depression. BMC Psychiatry. 2015;15:304. doi: 10.1186/s12888-015-0676-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Nordgreen T., Standal B., Mannes H., Haug T., Sivertsen B., Carlbring P., Andersson G., Heiervang E., Havik O.E. Guided self-help via internet for panic disorder: dissemination across countries. Comput. Hum. Behav. 2010;26(4):592–596. [Google Scholar]
  54. Nordgreen T., Haug T., Ost L.G., Andersson G., Carlbring P., Kvale G., Tangen T., Heiervang E., Havik O.E. Stepped care versus direct face-to-face cognitive behavior therapy for social anxiety disorder and panic disorder: a randomized effectiveness trial. Behav. Ther. 2016;47(2):166–183. doi: 10.1016/j.beth.2015.10.004. [DOI] [PubMed] [Google Scholar]
  55. Nordgreen T., Gjestad R., Andersson G., Carlbring P., Havik O.E. The implementation of guided Internet-based cognitive behaviour therapy for panic disorder in a routine-care setting: effectiveness and implementation efforts. Cogn. Behav. Ther. 2018;47(1):62–75. doi: 10.1080/16506073.2017.1348389. [DOI] [PubMed] [Google Scholar]
  56. Nordgreen, T., R. Gjestad, G. Andersson, P. Carlbring and O. E. Havik The effectiveness of guided internet-based cognitive behavioral therapy for social anxiety disorder in a routine care setting. Internet Interv. 13, 24–29. [DOI] [PMC free article] [PubMed]
  57. Nordmo M., Sinding A.I., Carlbring P., Andersson G., Havik O.E., Nordgreen T. Internet-delivered cognitive behavioural therapy with and without an initial face-to-face psychoeducation session for social anxiety disorder: a pilot randomized controlled trial. Internet Interv. 2015;2(4):429–436. [Google Scholar]
  58. Olthuis J.V., Watt M.C., Bailey K., Hayden J.A., Stewart S.H. Therapist-supported Internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst. Rev. 2016;3 doi: 10.1002/14651858.CD011565.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Proudfoot J., Clarke J., Birch M.R., Whitton A.E., Parker G., Manicavasagar V., Harrison V., Christensen H., Hadzi-Pavlovic D. Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial. BMC Psychiatry. 2013;13:312. doi: 10.1186/1471-244X-13-312. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Richards D.A., Borglin G. Implementation of psychological therapies for anxiety and depression in routine practice: two year prospective cohort study. J. Affect. Disord. 2011;133(1–2):51–60. doi: 10.1016/j.jad.2011.03.024. [DOI] [PubMed] [Google Scholar]
  61. Richards D., Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin. Psychol. Rev. 2012;32(4):329–342. doi: 10.1016/j.cpr.2012.02.004. [DOI] [PubMed] [Google Scholar]
  62. Richards D., Vigano N., O'Callaghan D.D., O'Brien E., Mooney J., Bonner C. Towards a gold standard for internet-delivered programs in behavioral and mental health. Eur. Psychiatry. 2016;33:S610. [Google Scholar]
  63. Ruwaard J., Kok R. Wild west eHealth: time to hold our horses? Eur. Health Psychol. 2015;17(1):45–49. [Google Scholar]
  64. Ruwaard J., Lange A., Schrieken B., Emmelkamp P. Efficacy and effectiveness of online cognitive behavioral treatment: a decade of interapy research. Stud. Health Technol. Inform. 2011;167:9–14. [PubMed] [Google Scholar]
  65. Ruwaard J., Lange A., Schrieken B., Dolan C.V., Emmelkamp P. The effectiveness of online cognitive behavioral treatment in routine clinical practice. PLoS One. 2012;7(7) doi: 10.1371/journal.pone.0040089. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Simblett S., Birch J., Matcham F., Yaguez L., Morris R. A systematic review and meta-analysis of e-mental health interventions to treat symptoms of posttraumatic stress. JMIR Ment. Health. 2017;4(2) doi: 10.2196/mental.5558. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Spence J., Titov N., Johnston L., Jones M.P., Dear B.F., Solley K. Internet-based trauma-focused cognitive behavioural therapy for PTSD with and without exposure components: a randomised controlled trial. J. Affect. Disord. 2014;162:73–80. doi: 10.1016/j.jad.2014.03.009. [DOI] [PubMed] [Google Scholar]
  68. Staples L.G., Fogliati V.J., Dear B.F., Nielssen O., Titov N. Internet-delivered treatment for older adults with anxiety and depression: implementation of the Wellbeing Plus Course in routine clinical care and comparison with research trial outcomes. BJPsych Open. 2016;2(5):307–313. doi: 10.1192/bjpo.bp.116.003400. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Titov N., Dear B.F., Johnston L., Lorian C., Zou J., Wootton B., Spence J., McEvoy P.M., Rapee R.M. Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: randomised controlled trial. PLoS One. 2013;8(7) doi: 10.1371/journal.pone.0062873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Titov N., Dear B.F., Staples L.G., Bennett-Levy J., Klein B., Rapee R.M., Shann C., Richards D., Andersson G., Ritterband L., Purtell C., Bezuidenhout G., Johnston L., Nielssen O.B. MindSpot clinic: an accessible, efficient, and effective online treatment service for anxiety and depression. Psychiatr. Serv. 2015;66(10):1043–1050. doi: 10.1176/appi.ps.201400477. [DOI] [PubMed] [Google Scholar]
  71. Titov N., Dear B.F., Staples L.G., Terides M.D., Karin E., Sheehan J., Johnston L., Gandy M., Fogliati V.J., Wootton B.M., McEvoy P.M. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided treatment for major depressive disorder and comorbid anxiety disorders: a randomized controlled trial. J. Anxiety Disord. 2015;35:88–102. doi: 10.1016/j.janxdis.2015.08.002. [DOI] [PubMed] [Google Scholar]
  72. Titov N., Dear B.F., Staples L.G., Bennett-Levy J., Klein B., Rapee R.M., Andersson G., Purtell C., Bezuidenhout G., Nielssen O.B. The first 30 months of the MindSpot Clinic: evaluation of a national e-mental health service against project objectives. Aust. N. Z. J. Psychiatry. 2017;51(12):1227–1239. doi: 10.1177/0004867416671598. [DOI] [PubMed] [Google Scholar]
  73. Titov N., Hadjistavropoulos H., Nordgreen T., Kaldo V., Mathiasen K. 9th Scientific Meeting of the International Society for Research into Internet Interventions, Berlin, Germany. 2017. Models of ICBT in routine care across five countries. [Google Scholar]
  74. Twomey C., O'Reilly G., Byrne M. Effectiveness of cognitive behavioural therapy for anxiety and depression in primary care: a meta-analysis. Fam. Pract. 2015;32(1):3–15. doi: 10.1093/fampra/cmu060. [DOI] [PubMed] [Google Scholar]
  75. Wang P.S., Aguilar-Gaxiola S., Alonso J., Angermeyer M.C., Borges G., Bromet E.J., Bruffaerts R., de Girolamo G., de Graaf R., Gureje O., Haro J.M., Karam E.G., Kessler R.C., Kovess V., Lane M.C., Lee S., Levinson D., Ono Y., Petukhova M., Posada-Villa J., Seedat S., Wells J.E. Use of mental health services for anxiety, mood, and substance disorders in 17 countries in the WHO world mental health surveys. Lancet. 2007;370(9590):841–850. doi: 10.1016/S0140-6736(07)61414-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  76. Whiteford H.A., Buckingham W.J., Harris M.G., Burgess P.M., Pirkis J.E., Barendregt J.J., Hall W.D. Estimating treatment rates for mental disorders in Australia. Aust. Health Rev. 2014;38(1):80–85. doi: 10.1071/AH13142. [DOI] [PubMed] [Google Scholar]
  77. Williams A.D., Andrews G. The effectiveness of Internet cognitive behavioural therapy (iCBT) for depression in primary care: a quality assurance study. PLoS One. 2013;8(2) doi: 10.1371/journal.pone.0057447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  78. Wootton B.M., Dear B.F., Johnston L., Terides M.D., Titov N. Remote treatment of obsessive-compulsive disorder: a randomized controlled trial. J. Obsessive Compuls. Relat. Disord. 2013;2(4):375–384. [Google Scholar]
  79. Wootton B.M., Dear B.F., Johnston L., Terides M.D., Titov N. Self-guided internet administered treatment for obsessive-compulsive disorder: results from two open trials. J. Obsessive Compuls. Relat. Disord. 2014;3(2):102–108. [Google Scholar]
  80. Wootton B.M., Dear B.F., Johnston L., Terides M.D., Titov N. Self-guided internet-delivered cognitive behavior therapy (iCBT) for obsessive–compulsive disorder: 12 month follow-up. Internet Interv. 2015;2(3):243–247. [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES