Skip to main content
PMC home page
Journal of Medical Internet Research logoLink to Journal of Medical Internet Research
. 2020 Aug 12;22(8):e18033. doi: 10.2196/18033

Views of Implementers and Nonimplementers of Internet-Administered Cognitive Behavioral Therapy for Depression and Anxiety: Survey of Primary Care Decision Makers in Sweden

Anders Brantnell 1,2,✉, Joanne Woodford 1, Enrico Baraldi 2, Theo van Achterberg 3, Louise von Essen 1
Editor: Gunther Eysenbach
Reviewed by: Heather Hadjistavropoulos, Daniela Austin, Kim Mathiasen
PMCID: PMC7450364  PMID: 32784186

Abstract

Background

Internet-administered cognitive behavioral therapy (ICBT) has been demonstrated to be an effective intervention for adults with depression and/or anxiety and is recommended in national guidelines for provision within Swedish primary care. However, the number and type of organizations that have implemented ICBT within primary care in Sweden is currently unclear. Further, there is a lack of knowledge concerning barriers and facilitators to ICBT implementation.

Objective

The two primary objectives were to identify and describe primary care organizations providing ICBT in Sweden and compare decision makers’ (ie, directors of primary care organizations) views on barriers and facilitators to implementation of ICBT among ICBT implementers (ie, organizations that offered ICBT) and nonimplementers (ie, organizations that did not offer ICBT).

Methods

An online survey based on a checklist for identifying barriers and facilitators to implementation was developed and made accessible to decision makers from all primary care organizations in Sweden. The survey consisted of background questions (eg, provision of ICBT and number of persons working with ICBT) and barriers and facilitators relating to the following categories: users, therapists, ICBT programs, organizations, and wider society.

Results

The participation rate was 35.75% (404/1130). The majority (250/404, 61.8%) of participants were health care center directors and had backgrounds in nursing. Altogether, 89.8% (363/404) of the participating organizations provided CBT. A minority (83/404, 20.5%) of organizations offered ICBT. Most professionals delivering ICBT were psychologists (67/83, 80%) and social workers (31/83, 37%). The majority (61/83, 73%) of organizations had 1 to 2 persons delivering ICBT interventions. The number of patients treated with ICBT during the last 12 months was 1 to 10 in 65% (54/83) of the organizations, ranging between 1 and 400 treated patients across the whole sample. There were 9 significant (P<.05) differences out of 37 possible between implementers and nonimplementers. For example, more implementers (48/51, 94%) than nonimplementers (107/139, 76.9%) perceived few technical problems (P<.001), and more implementers (53/77, 68%) than nonimplementers (103/215, 47.9%) considered that their organization has resources to offer ICBT programs (P<.001).

Conclusions

Despite research demonstrating the effectiveness of ICBT for depression and anxiety and national guidelines recommending its use, ICBT is implemented in few primary care organizations in Sweden. Several interesting differences between implementers and nonimplementers were identified, which may help inform interventions focusing on facilitating the implementation of ICBT.

Keywords: mental health, internet-administered CBT, self-management, implementation, barriers and facilitators, decision-making, eHealth, primary care

Introduction

Background

According to the World Health Organization, over 300 million people suffer from depression worldwide [1]. Further, over 260 million people suffer from anxiety disorders (eg, generalized anxiety disorder, panic disorder, and social anxiety disorder) [1]. Both depression and anxiety result in individual suffering and are associated with increased costs for both individuals and wider society [2-5]. One evidence-based intervention for depression and anxiety disorders (hereafter anxiety) is cognitive behavioral therapy (CBT), traditionally delivered face to face by a therapist [6-9]. However, despite the evidence base for CBT, a mental health treatment gap remains globally [10,11]. Some reasons for this treatment gap concern the fact that therapy is resource-intensive (eg, needing facilities to meet patients, requiring patients to travel to clinics, and needing trained, competent, and expensive therapists) [12]. One potential solution to overcome this treatment gap is the provision of psychological interventions via information technology and new media, referred to as e-mental health [13,14]. Internet-administered CBT (ICBT) [15-18] is a form of e-mental health that has been demonstrated to be as effective as face-to-face CBT [19] and may represent a cost-effective solution, which may increase the availability of evidence-based psychological interventions [20,21].

Sweden was among the first countries to conduct research on the efficacy of ICBT [16,22-25], leading to the introduction of national guidelines regarding the treatment of depression and anxiety [26]. These national guidelines are directed to decision makers at the level of primary care and offer health care providers with evidence-based intervention recommendations, including the provision of CBT and ICBT [26]. Specifically, national guidelines recommend ICBT to be offered to adults with mild to moderate symptoms of depression and anxiety at the primary care level [26]. The guidelines were recently revised to state health care providers should be able to choose the mode of CBT delivery by themselves, with recommended modes of delivery being individual, group, or internet-administered [27].

However, guideline introduction and uptake is difficult, and guidelines are not implemented into routine care automatically [28-32]. While barriers and facilitators to implementation in health care more generally have been identified and related to patients, informal caregivers, practitioners, health care organizations, and wider society [33], few studies have focused on barriers and facilitators to ICBT implementation more specifically [34-36]. Indeed, studies on ICBT implementation into clinical practice focus mostly on effectiveness and clinical feasibility [20,37-41]. Despite the current literature on ICBT implementation being in its infancy, recent studies have begun to identify barriers and facilitators. For example, evidence supporting ICBT programs is reported to be a key facilitator to implementation [34,35], whereas lack of resources [34] and lack of integration with the mental health care system [36] are reported to be important barriers to ICBT implementation. Implementation of e-mental health has received some interest, and studies have identified barriers such as lack of therapists’ knowledge on e-mental health [42] and lack of evidence-based programs [43]. Further, a recent systematic review of barriers and facilitators to implementation included 47 studies and identified factors such as patient and professional acceptance of e-mental health and fit with existing technologies [44]. In addition, while implementation of e-health interventions more generally is well covered in existing research [45-51], there have been recent calls to place more focus on implementation of e-health [52] to enable health care organizations to benefit from the promises of e-health solutions [53].

Studies examining the implementation of ICBT in clinical practice generally focus on the perspectives of patients and practitioners [20,37,39,41]. More recent studies have also included health care managers [34,36]. Although these three groups are important for successful implementation, these studies do not consider the formal decision to implement, which is usually made by primary care and specialized care directors. Consequently, research considering important stakeholders (ie, decision makers directly affecting the implementation of ICBT and their corresponding health care organizations) is lacking. More precisely, at present there is a lack of knowledge about the opinions of key decision makers regarding barriers and facilitators to implementation and which organizations have implemented ICBT in Sweden. Further, while some research has focused on decision makers operating at systems and national level (ie, policy makers and academic researchers) [43], there is a lack of research on decision makers closer to the implementation context and the health care setting such as primary care organization directors.

Aims and Objectives

This study has two main objectives: (1) identify and describe the primary care organizations providing ICBT in Sweden and (2) compare decision makers’ (ie, directors of primary care organizations) views on barriers and facilitators for the implementation of ICBT in organizations that have implemented ICBT (ie, are offering ICBT [implementers]) and have not implemented ICBT (ie, are not offering ICBT [nonimplementers]). In implementation research, it is common to distinguish between diffusion (ie, passive spread of innovations), dissemination (ie, active efforts to convince an organization to adopt and innovation), implementation (ie, active efforts to offer an innovation and integrate it within the organization), and sustainability (ie, making an innovation part of routine care) [54]. The Swedish guidelines recommending ICBT are relatively new and thus it is unlikely ICBT would be part of routine care in organizations; the assumption is that those who offer ICBT are in the process of integrating ICBT. To this end, the term implementation is used to describe organizations that offer ICBT.

Methods

Study Design

An online self-report survey was conducted between February and May 2016 with decision makers in primary care organizations in Sweden. The Checklist for Reporting Results of Internet e-Surveys [55] was followed.

Setting

Sweden has 290 municipalities situated across 21 administrative units called regions, which are responsible for health care provision. Each region has several primary care organizations forming the basis of the Swedish health care system. According to the Swedish law on health care (Chapter 2, 6§), primary care is part of the open care system and should provide basic medical treatment, ongoing care, preventive measures, and rehabilitation in cases that do not require medical and technical resources or other specialized competence accessible at hospitals [56]. Primary care is thus the first point of care and from here patients can be referred to specialized care. A typical primary care organization employs medical doctors, nurses, physiotherapists, and psychologists and can thus treat many patients. The number of health care professionals employed at primary care organizations is related to the number of listed patients, which ranges from approximately 3000 to 30,000 per organization. Consequently, some primary care organizations have relatively few listed patients whereas others have very many.

Both private and public primary care organizations receive funding from regions and operate under the same conditions in terms of personnel competence, financial resources, opening hours, patient access, and adherence to national guidelines regarding care provision. For primary care organizations willing to implement ICBT, there are three main options available: (1) buying a license from a company for proprietary ICBT programs and delivering the support by themselves, (2) hiring a company to deliver ICBT including support, or (3) connecting to the Platform for Support and Care. The Platform for Support and Care is owned and run by a company (Inera) owned by the Swedish Association of Local Authorities and Regions (SALAR). The Platform for Support and Care offers ICBT programs developed by companies and/or research groups who in turn receive financial compensation when their programs are used. All programs delivered via the Platform for Support and Care undergo a careful procedure to ensure effectiveness and safety. When connecting to this platform, regions can hire companies to support ICBT or use their own therapists to support ICBT. Connecting to the Platform for Support and Care and gaining access to ICBT programs results in financial costs for the regions, such as costs for connecting to the platform and purchasing treatment programs either with or without therapist support. As such, access to ICBT programs and therapist support may differ depending on the financial resources of the health care organization. Further, access to ICBT comes at a financial cost, often per treatment contact with a therapist. For instance, in Stockholm each contact costs around €10 (US $11.22) per session.

Recruitment and Study Procedures

Decision makers were heads of Swedish primary care organizations identified via SALAR, which supports development and provision of health care, and the Inspectorate for Treatment and Care, responsible for monitoring Swedish health care. We compiled a list of 1156 primary care organizations and mailing addresses and sent invitations to the decision makers via regular mail (see Multimedia Appendix 1 for an English translation of the invitation). Invitations were sent to the organizations because we did not have a complete list of decision makers’ email addresses. Invitations included a link to the online survey administered using the web-based tool, SurveyMonkey, full study information, and participation number and password allowing potential participants to reach the survey. Informed consent was provided online via SurveyMonkey, after which potential participants could access the survey. No incentives were offered for survey completion.

To maximize response rates, reminders were used [57]. Decision makers not completing the survey within two weeks received up to two telephone reminders and an email (except those whose email addresses were not available). The reminder email consisted of study information, the online link to the survey, participation number and password, and a reminder that the survey should be answered within two weeks. The reminder email was also accompanied by the letter previously sent through regular mail. After two weeks, nonrespondent decision makers were reminded once more following the same procedure.

To avoid duplicate responses from decision makers, the SurveyMonkey function allowing only one answer per computer was selected. Respondents who did not complete the survey at one occasion were able to return to the survey at a later time. The survey could be submitted even if all items were not answered. Approval was granted by the ethical review board that reviews applications from Uppsala University, Sweden (application number 2015/461).

Measures

The study team developed an online survey consisting of 21 background questions and 37 items about 5 factors: (1) user, (2) therapist, (3) program, (4) organization, and (5) society, with a 7-point Likert scale (1 = strongly disagree, 7 = strongly agree) and options for “do not know” and “do not wish to answer.” The survey also included two open-ended questions about barriers and facilitators to implementation and space for additional comments. This qualitative data will be reported elsewhere. The survey was informed by the checklist of barriers and facilitators for improvement of health care practice by Flottorp et al [33] and modified based on knowledge of barriers and facilitators specific to ICBT implementation originating from studies on effectiveness and clinical feasibility available when the study was designed [20,21,38,58-60]. The survey was piloted by a panel of 8 persons with knowledge of ICBT who were not part of the study population, including former primary care directors and persons responsible for information technology and health care in certain Swedish regions. The SALAR database was used to recruit the panel. Panel members were asked to indicate whether the questions were easy to understand and to provide suggestions for improvements. Overall, panel members were positive toward survey content but had some suggestions for improvements (see Multimedia Appendix 2 for a detailed description of the development and a summary of the changes). Figure 1 displays the final overall survey structure, and Multimedia Appendix 3 provides an English translation of the survey.

Figure 1.

Figure 1

Open in a new tab

Survey structure.

Statistical Analysis

All calculations were performed in R version 3.5.2 (R Foundation for Statistical Computing).

Responders, Survey Reminders, and Reasons for Not Answering

Descriptive statistics were used to describe numbers and proportions concerning survey completeness, reminders, and reasons for not answering.

Characteristics of Study Sample and Organizational Characteristics

Study sample and organizational characteristics are described using descriptive statistics.

Representativeness of the Study Sample

As noncoverage (ie, the survey fails to include some parts of the population [61]) is common in survey research, the coverage of the obtained sample was examined. Data from three known background variables for the entire population were used (ie, organizational form, city size, and health care region) and described through descriptive statistics. A chi-square test for equality of proportions was used to examine the representativeness of the sample in terms of the entire population.

Barriers and Facilitators to Implementation of Internet-Administered Cognitive Behavioral Therapy

The potential difference regarding the number of agree answers between implementers and nonimplementers was examined using the chi-square test for equality of proportions. The chi-square tests included both the number of nonagree (Likert scale options 1, 2, 3, and 4) and agree (Likert scale options 5, 6, and 7) answers. It was assumed that if a decision maker answered that the organization did not offer CBT (Q6), they did not offer ICBT (Q12), and the respondent was therefore transferred directly from Q6 to Q22. To this end, nonimplementation was imputed for Q12 for the decision makers who answered that they did not offer CBT (Q6).

Results

Responders

A total of 1156 survey invitations were sent, of which 97.75% (1130/1156) were shown to be eligible. Noneligible answers that were excluded were duplicate answers (n=13), bankruptcy/closed down (n=10), and not a primary care organization (n=3). A total of 426 decision makers answered at least the first survey question, providing an indication of interest to participate rate of 37.69% (426/1130). Participation [43] was defined as answering questions 1 to 22 (ie, background questions). The participation rate was 35.75% (404/1130). Completeness [55] was defined as completion of the last item (Q58). The completeness rate was 31.94% (361/1130).

Reminders

A total of 5.4% (22/404) of participants (based on participation rate) responded without a reminder, 47.5% (192/404) responded after being reminded via phone and email, 46.5% (188/404) responded after being reminded via email, and 0.4% (2/404) could not be reminded.

Reasons for Not Responding

Of the decision makers who did not answer the survey, 17.6% (128/726) provided reasons for not responding. The most frequent reason was no time (n=67), followed by not relevant for us (n=17) and decision maker employment ending (n=14).

Characteristics of the Study Sample and Organizational Characteristics

The majority (250/404, 61.8%) of decision makers were health care center directors with a background in nursing. A total of 89.8% (363/404) of organizations provided CBT, and 30.1% (122/404) had tried ICBT. Only 20.5% (83/404) reported that they currently offered ICBT and were accordingly categorized as ICBT implementers. Among the professionals delivering ICBT, 80% (67/83) were psychologists (including intern psychologists and psychotherapists) and 37% (31/83) were social workers. The majority (61/83, 73%) of organizations had 1 to 2 persons delivering ICBT. ICBT was offered to 1 to 10 patients during the last 12 months in 65% (54/83) of the organizations. Only 2% (2/83) of organizations had offered ICBT to over 100 persons during the last 12 months. Access to ICBT seemed to depend upon referral by a CBT therapist (36/83, 43%) or general practitioner (25/83, 30%) within the organization (for details on all collected characteristics see Multimedia Appendix 4).

Representativeness of the Sample

To evaluate the coverage of the sample, data on three known demographic variables for the entire population of primary care organizations in Sweden were used: organizational type (public/private), localization in cities of different sizes (city size), and health care regions. Table 1 displays the distribution of these variables in the study sample and entire population. Data indicates the study sample includes different organizational forms, city sizes, and health care regions. There was a significant difference between distribution of respondents and nonrespondent in terms organizational form (P<.001) and health care region (P<.001). There was no significant difference between distribution of respondents and nonrespondents regarding city size (P=.24). Consequently, the sample was representative in terms of city size but not concerning organizational form and health care region.

Table 1.

Distribution of respondents in study sample and population.

Demographic Sample (n=404), n (%) Population, n/N (%)
Organizational form


Private 131 (32.4) 131/465 (28.1)

Public 273 (67.5) 273/665 (41.0)
City size


Small (<30,000) 148 (36.6) 148/382 (38.7)

Medium (30,000-100,000) 148 (36.6) 148/418 (35.4)

Large (>100,000) 108 (26.7) 108/330 (32.7)
Health care regiona


North 61 (15.0) 61/123 (49.5)

Uppsala-Örebro 85 (21.0) 85/231 (36.7)

Stockholm 64 (15.8) 64/208 (30.7)

West-East 57 (14.1) 57/130 (43.8)

South 68 (16.8) 68/222 (30.6)

West 69 (17.0) 69/216 (31.9)
Open in a new tab

aIn Sweden, 21 regions are organized into 6 health care regions that facilitate cooperation and strategic work between the 21 regions.

Barriers and Facilitators to Implementation of Internet-Administered Cognitive Behavioral Therapy

Likert Scale Options and Item Nonresponse

Tables 2 to 6 display the numbers and percentages of decision makers, both implementers and nonimplementers, who agreed (Likert scale options 5, 6, and 7) with items Q22 through Q58. The survey options “do not know” and “do not wish to answer” are treated as item nonresponse and also shown in Tables 2 to 6 [61]. Item nonresponse varied from 22% to 66%.

Table 2.

Implementers and nonimplementers agreeing to user-related items.

Variable Item agreement and nonresponse

No ICBTa agree n/sample size (%) ICBT agree n/sample size (%) P value Item nonresponseb (n=403)
Q22. Adults with depression and/or anxiety have the computer skills needed to use ICBT programs. 177/236 (75.0) 42/62 (67.7) .32 105 (26.0)
Q23. Adults with depression and/or anxiety are capable of working on their own with ICBT programs. 151/217 (69.5) 39/59 (66.1) .72 127 (31.5)
Q24. Adults with depression and/or anxiety have interest in ICBT programs. 71/156 (45.5) 29/60 (48.3) .86 188 (46.6)
Q25. Adherence increases when treatment is delivered online to adults with depression and/or anxiety. 46/113 (40.7) 16/36 (44.4) .84 254 (63.0)
Q26. The barrier to seek help for adults with depression and/or anxiety is decreased when care is online. 123/183 (67.2) 19/37 (51.3) .10 183 (45.4)
Q27. Adults with depression and/or anxiety prefer to give confidential information to a computer. 51/144 (35.4) 7/35 (2.0) .12 224 (55.5)
Q28. Adults with depression and/or anxiety in rural areas can be reached with ICBT programs. 208/243 (85.5) 53/64 (82.8) .72 96 (23.8)
Open in a new tab

aICBT: internet-administered cognitive behavioral therapy.

bItem nonresponse (do not know, do not wish to answer) of the entire sample (n=403).

Table 6.

Implementers and nonimplementers agreeing to society-related items.

Variable Item agreement and nonresponse

No ICBTa agree n/sample size (%) ICBT agree n/sample size (%) P value Item nonresponseb (n=361)
Q57. Legislation does not hinder the introduction of ICBT for adults with depression and/or anxiety. 128/143 (89.5) 48/53 (90.5) .99 165 (45.7)
Q58. Public opinion supports the introduction of online treatments for adults with depression and/or anxiety. 91/123 (73.9) 26/37 (70.2) .81 201 (55.6)
Open in a new tab

aICBT: internet-administered cognitive behavioral therapy.

bItem nonresponse (do not know, do not wish to answer) of the entire sample (n=361).

Users

There were no significant differences (P<.05) between implementers and nonimplementers in terms of agreeing to user-related items (Table 2).

Therapists

Three therapist-related items were significant (P<.05; Table 3). More implementers (57/71, 80%) than nonimplementers (108/166, 65.0%) believed that therapists treating adults with depression and/or anxiety are positive toward the ICBT programs (P=.03; Q29). More implementers (68/76, 89%) than nonimplementers (125/192, 65.1%) believed therapists treating adults with depression and/or anxiety have knowledge of the ICBT programs (P<.001; Q30). More implementers (55/62, 88%) than nonimplementers (103/148, 69.5%) also believed therapists treating adults with depression and/or anxiety have confidence in the guidelines recommending ICBT programs (P<.001; Q33).

Table 3.

Implementers and nonimplementers agreeing to therapist-related items.

Variable Item agreement and nonresponse

No ICBTa agree n/sample size (%) ICBT agree n/sample size (%) P value Item nonresponseb (n=396)
Q29. Therapists treating adults with depression and/or anxiety are positive toward the ICBT programs. 108/166 (65.0) 57/71 (80.2) .03 159 (40.1)
Q30. Therapists treating adults with depression and/or anxiety have knowledge of the ICBT programs. 125/192 (65.1) 68/76 (89.4) .001 128 (32.3)
Q31. Therapists treating adults with depression and/or anxiety only need a little training in ICBT programs. 94/140 (67.1) 31/52 (59.6) .42 204 (51.5)
Q32. Therapists treating adults with depression and/or anxiety have the computer skills needed for ICBT. 195/227 (85.9) 66/72 (91.6) .28 97 (24.4)
Q33. Therapists treating adults with depression and/or anxiety have confidence in the guidelines recommending ICBT programs. 103/148 (69.5) 55/62 (88.7) .001 186 (46.9)
Q34. Therapists treating adults with depression and/or anxiety can motivate patients to participate. 159/199 (79.8) 54/66 (81.8) .87 131 (33.0)
Q35. More therapists support the introduction of ICBT programs than oppose it. 89/119 (74.7) 40/49 (81.6) .45 228 (57.5)
Open in a new tab

aICBT: internet-administered cognitive behavioral therapy.

bItem nonresponse (do not know, do not wish to answer) of the entire sample (n=396).

Program

Two program-related items were significant (P<.05; Table 4). More nonimplementers (77/101, 76.2%) than implementers (15/32, 46%) believed that ICBT programs for adults with depression and/or anxiety offer alternative learning formats (P<.001; Q38). More implementers (48/51, 94%) than nonimplementers (107/139, 76.9%) in turn believed that ICBT programs for adults with depression and/or anxiety are not plagued with big technical problems (P<.001; Q39).

Table 4.

Implementers and nonimplementers agreeing to program-related items.

Variable Item agreement and nonresponse

No ICBTa agree n/sample size (%) ICBT agree n/sample size (%) P value Item nonresponseb (n=389)
Q36. ICBT programs for adults with depression and/or anxiety should come with a help desk for therapists. 164/198 (82.8) 41/45 (91.1) .25 146 (37.5)
Q37. ICBT programs are well suited for adults with depression and/or anxiety. 142/184 (77.1) 52/62 (83.8) .37 143 (36.7)
Q38. ICBT programs for adults with depression and/or anxiety offer alternative learning formats. 77/101 (76.2) 15/32 (46.8) .001 256 (65.8)
Q39. ICBT programs for adults with depression and/or anxiety are not plagued with big technical problems. 107/139 (76.9) 48/51 (94.1) .001 199 (51.1)
Q40. It should be possible to trial the ICBT programs. 222/247 (89.8) 45/55 (81.8) .15 87 (22.3)
Q41. It is possible to measure the effect on depression and/or anxiety when providing ICBT programs. 122/137 (89.0) 36/42 (85.7) .75 210 (53.9)
Q42. ICBT programs for adults with depression and/or anxiety are easy to use. 78/95 (82.1) 46/52 (88.4) .44 242 (62.2)
Q43. ICBT programs for adults with depression and/or anxiety can be integrated with the care structure. 161/208 (77.4) 63/73 (86.3) .15 108 (27.7)
Q44. ICBT programs for adults with depression and/or anxiety can replace face-to-face CBT. 58/202 (28.7) 18/62 (29.0) .99 125 (32.1)
Q45. It is easy to get access to ICBT programs for adults with depression and/or anxiety. 59/111 (53.1) 39/57 (68.4) .08 221 (56.8)
Q46. ICBT programs for adults with depression and/or anxiety are well grounded on research evidence. 95/117 (81.1) 39/49 (79.5) .98 223 (57.3)
Q47. GPsc referring adults with depression and/or anxiety to ICBT are positive toward ICBT programs. 88/130 (67.6) 30/46 (65.2) .90 213 (54.7)
Open in a new tab

aICBT: internet-administered cognitive behavioral therapy.

bItem nonresponse (do not know, do not wish to answer) of the entire sample (n=389).

cGPs: general practitioners.

Organization

Four organization-related items were significant (P<.05; Table 5). More implementers (53/77, 68%) than nonimplementers (103/215, 47.9%) considered that they have resources to offer ICBT programs (P<.001; Q48). More implementers (41/47, 87%) than nonimplementers (59/89, 66%) believed that contracts with service providers allow introduction of ICBT (P=.02; Q54). More implementers (36/72, 50%) than nonimplementers (35/226, 15.4%) considered that the concept of online treatment is well established at their organization (P<.001; Q55). More implementers (58/66, 87%) than nonimplementers (87/164, 53.0%) believed the patient referral process allows for the introduction of ICBT (P<.001; Q56).

Table 5.

Implementers and nonimplementers agreeing to organization-related items.

Variable Item agreement and nonresponse

No ICBTa agree n/sample size (%) ICBT agree n/sample size (%) P value Item nonresponseb (n=381)
Q48. Our organization has resources to offer ICBT programs to adults with depression and/or anxiety. 103/215 (47.9) 53/77 (68.8) .001 89 (23.3)
Q49. ICBT programs can decrease care costs in treatment of adults with depression and/or anxiety. 157/196 (80.1) 42/57 (73.6) .39 128 (33.5)
Q50. Existing information management systems allows administration of patients enrolled in ICBT. 57/105 (54.2) 34/54 (62.9) .38 222 (58.2)
Q51. Existing quality assurance and patient safety systems are compatible with the requirements to offer ICBT to adults with depression and/or anxiety. 56/85 (65.8) 35/44 (79.5) .16 252 (66.1)
Q52. Existing continuing education systems of therapists are compatible with the training to introduce ICBT to adults with depression and/or anxiety. 58/91 (63.7) 31/44 (70.4) .56 246 (64.5)
Q53. Internal regulations allow introduction of ICBT for adults with depression and/or anxiety. 123/152 (80.9) 64/71 (90.1) .12 158 (41.4)
Q54. Contracts with service providers allow introduction of ICBT for adults with depression and/or anxiety. 59/89 (66.2) 41/47 (87.2) .02 245 (64.3)
Q55. The concept of online treatment to adults with depression and/or anxiety is well established at our organization. 35/226 (15.4) 36/72 (50.0) .001 83 (21.7)
Q56. The patient referral process allows introduction of ICBT for adults with depression and/or anxiety. 87/164 (53.0) 58/66 (87.8) .001 151 (39.6)
Open in a new tab

aICBT: internet-administered cognitive behavioral therapy.

bItem nonresponse (do not know, do not wish to answer) of the entire sample (n=381).

Society

There were no significant differences (P<.05) between implementers and nonimplementers in terms of society-related items (Table 6).

Discussion

Principal Findings

The participation rate was 35.75% (404/1130). The majority (250/404, 61.8%) of participants were health care center directors with a background in nursing. A total of 89.9% (363/404) of the participating organizations provided CBT. A minority (83/404, 20.5%) provided ICBT and thus were implementers. In general, psychologists (67/83, 80%) and social workers (31/83, 37%) delivered ICBT to patients, and the majority (61/83, 73%) of the organizations had 1 to 2 persons delivering ICBT. There were 9 significant (P<.05) differences out of 37 possible between implementers and nonimplementers. For example, more implementers (68/76, 89%) than nonimplementers (125/192, 65.1%) considered that therapists treating adults with depression and/or anxiety have knowledge of the ICBT programs (P<.001), and more implementers (58/66, 87%) than nonimplementers (87/164, 53.0%) believed the patient referral process allows the introduction of ICBT (P<.001).

Although e-mental health initiatives are encouraged, and the Swedish national guidelines include provision of ICBT in Swedish primary care among their recommendations, few organizations in our sample provided ICBT. Moreover, in a majority of cases only 1 to 10 patients had been treated with ICBT during the last 12 months, and the majority of organizations only had 1 to 2 CBT therapists working with ICBT. The number of ICBT implementers is not as high as one would expect based on the strong recommendations and claimed benefits of ICBT, such as increased patient access and lower costs [21,22]. Previous research indicates the implementation rate of guidelines across various treatment areas has been between 50% and 70% [62-65]. However, implementation research does not provide detailed guidance concerning whether a 20% implementation rate could be considered low or high. Further, the implementation of a complex health care intervention, such as ICBT, is more demanding than implementing clinical guidelines in general. Indeed, our findings are in line with a recent study examining ICBT implementation among psychologists in the Netherlands where the implementation rate was around 16% [66].

One reason for the low implementation rate may pertain to not having a workforce who can support the provision of ICBT. Indeed, the majority of implementing organizations had only 1 to 2 therapists working with ICBT and CBT. Moreover, all organizations providing ICBT also provided CBT, and in cases ICBT and CBT rely on the same workforce, this workforce could be inadequate to support ICBT implementation. Further, CBT therapists may not represent the best solution, given they are a highly trained and expensive workforce, with a demand exceeding supply [67]. A possible solution could be to train mental health workers to support ICBT, as done by the Improving Access to Psychological Therapies program in England, whereby a workforce of psychological well-being practitioners was established to deliver low-intensity CBT [67]. Future research could explore the workforce aspect in Sweden, and current findings could be helpful for policymakers trying to promote the implementation of ICBT.

Our findings show that 94% (48/51) of implementers consider that ICBT is not plagued with technical problems, whereas only 76.9% (107/139) of nonimplementers share this opinion (Q39). Existing research on implementation is clear regarding technical problems, showing innovations low in complexity to be more easily adopted [68-70]. Another interesting finding is that 50% (36/72) of implementers believed that the concept of online treatment regarding depression and/or anxiety is well established at their organization compared with 15.4% (35/226) of nonimplementers who believed this be the case (Q55). The survey did not define the word concept, and thus this could have been interpreted differently by decision makers. However, as relatively many implementers compared with nonimplementers perceived the concept to be well established there seems to be a clear pattern in how this item is perceived by the decision makers. One possible reason for this finding could be that implementation of ICBT may increase understanding of online treatments and thus strengthen the concept.

Another interesting finding is that the majority of implementers (53/77, 68%) consider that they have resources to offer ICBT programs compared with a minority among nonimplementers (103/215, 47.9%; Q48), which indicates lack of resources may be an important barrier to implementation of ICBT in Swedish primary care. This factor is identified as central in existing ICBT implementation research [34] and in research concerning implementation of eHealth [45,71]. Findings also show that 80% (57/71) of implementers consider therapists treating adults with depression and/or anxiety are positive toward the ICBT programs compared with 65.0% (108/166) of nonimplementers (Q29). This finding implies therapist acceptance could be an important facilitator to implementation, in line with existing ICBT implementation research [21,60] and research concerning e-mental health [42,44]. Findings also demonstrate that 87% (41/47) of implementers believe contracts with service providers allow for the introduction of ICBT compared with 66% (59/89) of nonimplementers (Q54). Indeed, contracts with service providers are included in the framework regarding barriers and facilitators to implementation by Flottorp et al [33]. As such, our findings may imply lack of contracts with service providers could be a sizeable barrier to implementation of ICBT in Sweden.

While 28 barriers and facilitators to implementation did not show significant differences between implementers and nonimplementers, they still provide important findings. The majority (219/298, 73.4%) of decision makers considered users to possess the computer skills needed to use ICBT programs. Further, the majority (261/299, 87.2%) considered therapists to have the computer skills required to offer ICBT. Lack of computer skills among patients and therapists was found to be a barrier to implementation in wider research concerning implementation of e-mental health [43]. Further, therapists’ computer skills is reported to be a facilitator to implementation of eHealth [45]. Findings also show that the majority (134/166, 80.7%) of decision makers considered ICBT programs to be evidence-based. This factor is a key facilitator in ICBT implementation [34,35] and an important barrier, if absent, in e-mental health [43] and eHealth implementation [47,48]. Further, the majority (199/253, 78.6%) of decision makers considered that ICBT can decrease costs related to providing care, which is also suggested in existing studies concerning eHealth programs [46,47].

Limitations

First, this study aimed to cover all primary care organizations in Sweden. However, recruitment of decision makers was difficult and thus the study only includes a limited sample of primary care organizations. However, participation rate was 35.75% (404/1130), which is in line with a recent study examining the implementation of ICBT from therapists’ point of view [66] and the sample was representative of the population on city size. Second, we cannot exclude the possibility that people who were interested in ICBT were more inclined to respond and thus our sample may include a larger proportion of implementers than nonimplementers compared with the entire population. Third, we focused on decision makers in Swedish primary care, who are often not experts in ICBT. However, decision makers decide whether or not to implement and thus their perceptions of barriers and facilitators are key for understanding the chances of successful implementation. Future studies may also include the views of Swedish primary care therapists representing implementing organizations and could compare the views of therapists with those of decision makers. Fourth, this study presents results of a survey focusing on a set of factors and whether respondents agreed on their importance on a quantitative scale but not to what extent each factor represented a barrier or facilitator to the implementation process. Future studies combining both survey and semistructured interview data may provide a more detailed exploration of barriers and facilitators to implementation and aid interpretation of survey findings.

Fifth, the survey did not explore whether implementing organizations used specific implementation strategies associated with improved changes to existing practice [29]—for example, opinion leaders or audit and feedback to facilitate implementation [72]. Nor did the survey explore to what extent decision makers considered ICBT implementation a success (eg, in terms of increased ICBT use, costs, and resources). Future research into barriers and facilitators of ICBT implementation may examine ICBT implementation strategies used and the perceived success of these strategies. Sixth, our results are influenced by multiple testing: with alpha .05 and 37 tests, 2 of the 9 differences are likely to be produced by chance alone. Seventh, nonresponse per item was relatively high (varied between 22% and 66%), which could impact the validity of the results. However, we observe that nonresponse is relatively evenly distributed between items and thus a reasonable assumption is that nonresponse does not impact validity.

Despite the aforementioned limitations, to our knowledge, this is the first study to provide data on the implementation of ICBT in one of the countries that pioneered ICBT research and treatment, elucidating the views of decision makers on the implementation of ICBT in primary care.

Conclusions

Despite existing scientific evidence supporting implementation of ICBT in primary care and guidelines recommending implementation, most primary care organizations in Sweden still only offer traditional CBT. This study provides an overview of the characteristics of implementers in one of the countries that pioneered research in ICBT, identifying interesting differences in terms of perceived barriers and facilitators between implementers and nonimplementers that may inform future implementation interventions to improve the routine uptake of ICBT in Swedish primary care but also in other countries introducing ICBT.

Acknowledgments

The authors thank all decision makers in primary care who participated in the study. They also thank SALAR for their advice concerning ensuring the survey was more suitable for decision makers in Swedish primary care. The authors also thank several colleagues for their input and support. This work was supported by U-CARE, which is a strategic research environment funded by the Swedish Research Council (dnr 2009-1093). The funder did not contribute to design and conducting of the study and did not participate in analysis of the results.

Abbreviations

CBT

cognitive behavioral therapy

ICBT

internet-administered cognitive behavioral therapy

SALAR

Swedish Association of Local Authorities and Regions

Appendix

Multimedia Appendix 1

English translation of invitation letter used to recruit respondents.

Multimedia Appendix 2

Survey development.

Multimedia Appendix 3

English translation of the online questionnaire as it was presented to the respondents.

Multimedia Appendix 4

Characteristics of the study sample and organizational characteristics.

Footnotes

Conflicts of Interest: None declared.

References

  • 1.Depression and other common mental disorders: global health estimates. Geneva: World Health Organization; 2017. [2020-06-17]. https://apps.who.int/iris/bitstream/handle/10665/254610/WHO-MSD-MER-2017.2-eng.pdf?sequence=1. [Google Scholar]
  • 2.Lewis C, Pearce J, Bisson JI. Efficacy, cost-effectiveness and acceptability of self-help interventions for anxiety disorders: systematic review. Br J Psychiatry. 2012 Jan;200(1):15–21. doi: 10.1192/bjp.bp.110.084756. http://bjp.rcpsych.org/cgi/pmidlookup?view=long&pmid=22215865. [DOI] [PubMed] [Google Scholar]
  • 3.Stuhldreher N, Leibing E, Leichsenring F, Beutel ME, Herpertz S, Hoyer J, Konnopka A, Salzer S, Strauss B, Wiltink J, König H. The costs of social anxiety disorder: the role of symptom severity and comorbidities. J Affect Disord. 2014 Aug;165:87–94. doi: 10.1016/j.jad.2014.04.039. [DOI] [PubMed] [Google Scholar]
  • 4.Newman MG, Llera SJ, Erickson TM, Przeworski A, Castonguay LG. Worry and generalized anxiety disorder: a review and theoretical synthesis of evidence on nature, etiology, mechanisms, and treatment. Annu Rev Clin Psychol. 2013;9:275–297. doi: 10.1146/annurev-clinpsy-050212-185544. http://europepmc.org/abstract/MED/23537486. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ferrari AJ, Charlson FJ, Norman RE, Patten SB, Freedman G, Murray CJL, Vos T, Whiteford HA. Burden of depressive disorders by country, sex, age, and year: findings from the global burden of disease study 2010. PLoS Med. 2013 Nov;10(11):e1001547. doi: 10.1371/journal.pmed.1001547. http://dx.plos.org/10.1371/journal.pmed.1001547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Cuijpers P, Cristea IA, Karyotaki E, Reijnders M, Huibers MJH. How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence. World Psychiatry. 2016 Oct;15(3):245–258. doi: 10.1002/wps.20346. doi: 10.1002/wps.20346. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, Dobson KS. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry. 2013 Jul;58(7):376–385. doi: 10.1177/070674371305800702. [DOI] [PubMed] [Google Scholar]
  • 8.Olatunji BO, Cisler JM, Deacon BJ. Efficacy of cognitive behavioral therapy for anxiety disorders: a review of meta-analytic findings. Psychiatr Clin North Am. 2010 Sep;33(3):557–577. doi: 10.1016/j.psc.2010.04.002. [DOI] [PubMed] [Google Scholar]
  • 9.Hofmann SG, Asnaani A, Vonk IJJ, Sawyer AT, Fang A. The efficacy of cognitive behavioral therapy: a review of meta-analyses. Cognit Ther Res. 2012 Oct 1;36(5):427–440. doi: 10.1007/s10608-012-9476-1. http://europepmc.org/abstract/MED/23459093. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, Al-Hamzawi A, Alonso J, Andrade L, Borges G, Bruffaerts R, Bunting B, Florescu S, Gureje O, Haro JM, He Y, Hinkov H, Karam E, Kawakami N, Lee S, Navarro-Mateu F, Piazza M, Posada-Villa J, Kessler RC. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry. 2017 Dec;210(2):119–124. doi: 10.1192/bjp.bp.116.188078. doi: 10.1192/bjp.bp.116.188078. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Alonso J, Liu Z, Evans-Lacko S, Sadikova E, Sampson N, Chatterji S, Abdulmalik J, Aguilar-Gaxiola S, Al-Hamzawi A, Andrade LH, Bruffaerts R, Cardoso G, Cia A, Florescu S, de Girolamo G, Gureje O, Haro JM, He Y, de Jonge P, Karam EG, Kawakami N, Kovess-Masfety V, Lee S, Levinson D, Medina-Mora ME, Navarro-Mateu F, Pennell B, Piazza M, Posada-Villa J, Ten Have M, Zarkov Z, Kessler RC, Thornicroft G, WHO World Mental Health Survey Collaborators Treatment gap for anxiety disorders is global: results of the World Mental Health Surveys in 21 countries. Depress Anxiety. 2018 Mar;35(3):195–208. doi: 10.1002/da.22711. http://europepmc.org/abstract/MED/29356216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Kazdin AE, Blase SL. The efficacy of cognitive behavioral therapy: a review of meta-analyses. Perspect Psychol Sci. 2011 Jan;6(1):21–37. doi: 10.1177/1745691610393527. [DOI] [PubMed] [Google Scholar]
  • 13.Riper H, Andersson G, Christensen H, Cuijpers P, Lange A, Eysenbach G. Theme issue on e-mental health: a growing field in internet research. J Med Internet Res. 2010;12(5):e74. doi: 10.2196/jmir.1713. http://www.jmir.org/2010/5/e74/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Musiat P, Tarrier N. Collateral outcomes in e-mental health: a systematic review of the evidence for added benefits of computerized cognitive behavior therapy interventions for mental health. Psychol Med. 2014 Nov;44(15):3137–3150. doi: 10.1017/S0033291714000245. [DOI] [PubMed] [Google Scholar]
  • 15.Andersson G. Internet interventions: past, present and future. Internet Interv. 2018 Jun;12:181–188. doi: 10.1016/j.invent.2018.03.008. https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(18)30015-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. 2014 Oct;13(3):288–295. doi: 10.1002/wps.20151. doi: 10.1002/wps.20151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Johansson R, Andersson G. Internet-based psychological treatments for depression. Expert Rev Neurother. 2012 Jul;12(7):861–869. doi: 10.1586/ern.12.63. [DOI] [PubMed] [Google Scholar]
  • 18.Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012 Jun;32(4):329–342. doi: 10.1016/j.cpr.2012.02.004. [DOI] [PubMed] [Google Scholar]
  • 19.Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. 2018 Jan;47(1):1–18. doi: 10.1080/16506073.2017.1401115. [DOI] [PubMed] [Google Scholar]
  • 20.Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry. 2014 Feb;13(1):4–11. doi: 10.1002/wps.20083. doi: 10.1002/wps.20083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Andersson G. The promise and pitfalls of the internet for cognitive behavioral therapy. BMC Med. 2010;8:82. doi: 10.1186/1741-7015-8-82. http://www.biomedcentral.com/1741-7015/8/82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196–205. doi: 10.1080/16506070903318960. [DOI] [PubMed] [Google Scholar]
  • 23.Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev. 2006 Jan;26(1):17–31. doi: 10.1016/j.cpr.2005.07.003. [DOI] [PubMed] [Google Scholar]
  • 24.Sjöström J, von Essen L, Grönqvist H. The origin and impact of ideals in ehealth research: experiences from the U-CARE research environment. JMIR Res Protoc. 2014 May 23;3(2):e28. doi: 10.2196/resprot.3202. http://www.researchprotocols.org/2014/2/e28/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007 Mar;37(3):319–328. doi: 10.1017/S0033291706008944. [DOI] [PubMed] [Google Scholar]
  • 26.Socialstyrelsen . Nationella Riktlinjer för Vård vid Depression och Ångestsyndrom. Stockholm: Socialstyrelsen; 2010. [2020-06-22]. http://www.sfbup.se/wp-content/uploads/2016/11/2010-3-4.pdf. [Google Scholar]
  • 27.Socialstyrelsen . Nationella Riktlinjer för Vård vid Depression och Ångestsyndrom. Stockholm: Socialstyrelsen; 2017. [2020-06-22]. https://www.socialstyrelsen.se/globalassets/sharepoint-dokument/artikelkatalog/nationella-riktlinjer/2017-12-4.pdf. [Google Scholar]
  • 28.Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004 Feb;8(6):1–72. doi: 10.3310/hta8060. http://www.journalslibrary.nihr.ac.uk/hta/volume-8/issue-6. [DOI] [PubMed] [Google Scholar]
  • 29.Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012 May 31;7:50. doi: 10.1186/1748-5908-7-50. https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-7-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Wang Z, Norris SL, Bero L. The advantages and limitations of guideline adaptation frameworks. Implement Sci. 2018 May 29;13(1):72. doi: 10.1186/s13012-018-0763-4. https://implementationscience.biomedcentral.com/articles/10.1186/s13012-018-0763-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Liang L, Bernhardsson S, Vernooij RWM, Armstrong MJ, Bussières A, Brouwers MC, Gagliardi AR, Members of the Guidelines International Network Implementation Working Group Use of theory to plan or evaluate guideline implementation among physicians: a scoping review. Implement Sci. 2017 Feb 27;12(1):26. doi: 10.1186/s13012-017-0557-0. https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0557-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.DeJonckheere M, Robinson CH, Evans L, Lowery J, Youles B, Tremblay A, Kelley C, Sussman JB. Designing for clinical change: creating an intervention to implement new statin guidelines in a primary care clinic. JMIR Hum Factors. 2018 Apr 24;5(2):e19. doi: 10.2196/humanfactors.9030. https://humanfactors.jmir.org/2018/2/e19/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013 Mar 23;8:35. doi: 10.1186/1748-5908-8-35. https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-8-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Hadjistavropoulos HD, Nugent MM, Dirkse D, Pugh N. Implementation of internet-delivered cognitive behavior therapy within community mental health clinics: a process evaluation using the consolidated framework for implementation research. BMC Psychiatry. 2017 Sep 12;17(1):331. doi: 10.1186/s12888-017-1496-7. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-017-1496-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Titov N, Dear B, Nielssen O, Staples L, Hadjistavropoulos H, Nugent M, Adlam K, Nordgreen T, Bruvik KH, Hovland A, Repål A, Mathiasen K, Kraepelien M, Blom K, Svanborg C, Lindefors N, Kaldo V. ICBT in routine care: a descriptive analysis of successful clinics in five countries. Internet Interv. 2018 Sep;13:108–115. doi: 10.1016/j.invent.2018.07.006. https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(18)30037-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Folker AP, Mathiasen K, Lauridsen SM, Stenderup E, Dozeman E, Folker MP. Implementing internet-delivered cognitive behavior therapy for common mental health disorders: a comparative case study of implementation challenges perceived by therapists and managers in five European internet services. Internet Interv. 2018 Mar;11:60–70. doi: 10.1016/j.invent.2018.02.001. https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(17)30120-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Nordgreen T, Gjestad R, Andersson G, Carlbring P, Havik OE. The implementation of guided Internet-based cognitive behaviour therapy for panic disorder in a routine-care setting: effectiveness and implementation efforts. Cogn Behav Ther. 2018 Jan;47(1):62–75. doi: 10.1080/16506073.2017.1348389. [DOI] [PubMed] [Google Scholar]
  • 38.Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med. 2009 May;39(5):705–712. doi: 10.1017/S0033291708004224. [DOI] [PubMed] [Google Scholar]
  • 39.Hedman E, Ljótsson B, Rück C, Bergström J, Andersson G, Kaldo V, Jansson L, Andersson E, Andersson E, Blom K, Falk L, Ivarsson J, Nasri B, Rydh S, Lindefors N. Effectiveness of internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatr Scand. 2013 Dec;128(6):457–467. doi: 10.1111/acps.12079. [DOI] [PubMed] [Google Scholar]
  • 40.Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res. 2012 Dec;12(6):745–764. doi: 10.1586/erp.12.67. [DOI] [PubMed] [Google Scholar]
  • 41.Hedman E, Ljótsson B, Kaldo V, Hesser H, Kraepelien M, Andersson E, Rück C, Svanborg C, Andersson G, Lindefors N. Effectiveness of Internet-based cognitive behaviour therapy for depression in routine psychiatric care. J Affect Disord. 2014 Feb;155:49–58. doi: 10.1016/j.jad.2013.10.023. [DOI] [PubMed] [Google Scholar]
  • 42.Feijt MA, de Kort YA, Bongers IM, IJsselsteijn WA. Perceived drivers and barriers to the adoption of emental health by psychologists: the construction of the levels of adoption of emental health model. J Med Internet Res. 2018 Dec 24;20(4):e153. doi: 10.2196/jmir.9485. http://www.jmir.org/2018/4/e153/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Wozney L, Newton AS, Gehring ND, Bennett K, Huguet A, Hartling L, Dyson MP, McGrath P. Implementation of eMental Health care: viewpoints from key informants from organizations and agencies with eHealth mandates. BMC Med Inform Decis Mak. 2017 Jun 02;17(1):78. doi: 10.1186/s12911-017-0474-9. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-017-0474-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Vis C, Mol M, Kleiboer A, Bührmann L, Finch T, Smit J, Riper H. Improving implementation of emental health for mood disorders in routine practice: systematic review of barriers and facilitating factors. JMIR Ment Health. 2018 Mar 16;5(1):e20. doi: 10.2196/mental.9769. https://mental.jmir.org/2018/1/e20/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) Implement Sci. 2016 Oct 26;11(1):146. doi: 10.1186/s13012-016-0510-7. https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0510-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Elbert NJ, van Os-Medendorp H, van Renselaar W, Ekeland AG, Hakkaart-van Roijen L, Raat H, Nijsten TEC, Pasmans SGMA. Effectiveness and cost-effectiveness of ehealth interventions in somatic diseases: a systematic review of systematic reviews and meta-analyses. J Med Internet Res. 2014;16(4):e110. doi: 10.2196/jmir.2790. http://www.jmir.org/2014/4/e110/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Mileski M, Kruse CS, Catalani J, Haderer T. Adopting telemedicine for the self-management of hypertension: systematic review. JMIR Med Inform. 2017 Oct 24;5(4):e41. doi: 10.2196/medinform.6603. http://medinform.jmir.org/2017/4/e41/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Ariens LF, Schussler-Raymakers FM, Frima C, Flinterman A, Hamminga E, Arents BW, Bruijnzeel-Koomen CA. Barriers and facilitators to ehealth use in daily practice: perspectives of patients and professionals in dermatology. J Med Internet Res. 2017 Sep 05;19(9):e300. doi: 10.2196/jmir.7512. http://www.jmir.org/2017/9/e300/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Lennon MR, Bouamrane M, Devlin AM, O'Connor S, O'Donnell C, Chetty U, Agbakoba R, Bikker A, Grieve E, Finch T, Watson N, Wyke S, Mair FS. Readiness for delivering digital health at scale: lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. J Med Internet Res. 2017 Feb 16;19(2):e42. doi: 10.2196/jmir.6900. http://www.jmir.org/2017/2/e42/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Swinkels ICS, Huygens MWJ, Schoenmakers TM, Oude Nijeweme-D'Hollosy W, van Velsen L, Vermeulen J, Schoone-Harmsen M, Jansen YJ, van Schayck OC, Friele R, de Witte L. Lessons learned from a living lab on the broad adoption of ehealth in primary health care. J Med Internet Res. 2018 Mar 29;20(3):e83. doi: 10.2196/jmir.9110. http://www.jmir.org/2018/3/e83/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.van der Meij E, Huirne JA, Ten Cate AD, Stockmann HB, Scholten PC, Davids PH, Bonjer HJ, Anema JR. A perioperative ehealth program to enhance postoperative recovery after abdominal surgery: process evaluation of a randomized controlled trial. J Med Internet Res. 2018 Jan 02;20(1):e1. doi: 10.2196/jmir.8338. http://www.jmir.org/2018/1/e1/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Buis L. Implementation: the next giant hurdle to clinical transformation with digital health. J Med Internet Res. 2019 Nov 20;21(11):e16259. doi: 10.2196/16259. https://www.jmir.org/2019/11/e16259/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Kukafka R. Digital health consumers on the road to the future. J Med Internet Res. 2019 Nov 21;21(11):e16359. doi: 10.2196/16359. https://www.jmir.org/2019/11/e16359/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. http://europepmc.org/abstract/MED/15595944. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) J Med Internet Res. 2004 Sep 29;6(3):e34. doi: 10.2196/jmir.6.3.e34. http://www.jmir.org/2004/3/e34/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Socialdepartementet . Hälso-och Sjukvårdslag SFS 2017:30. Stockholm: Socialdepartementet; 2017. [2020-06-22]. https://www.riksdagen.se/sv/dokument-lagar/dokument/svensk-forfattningssamling/halso--och-sjukvardslag_sfs-2017-30. [Google Scholar]
  • 57.Shih T, Xitao Fan Comparing response rates from web and mail surveys: a meta-analysis. Field Methods. 2008 Mar 03;20(3):249–271. doi: 10.1177/1525822X08317085. [DOI] [Google Scholar]
  • 58.Andrews G, Titov N. Hit and miss: innovation and the dissemination of evidence based psychological treatments. Behav Res Ther. 2009 Nov;47(11):974–979. doi: 10.1016/j.brat.2009.07.007. [DOI] [PubMed] [Google Scholar]
  • 59.Gega L, Marks I, Mataix-Cols D. Computer-aided CBT self-help for anxiety and depressive disorders: experience of a London clinic and future directions. J Clin Psychol. 2004 Feb;60(2):147–157. doi: 10.1002/jclp.10241. [DOI] [PubMed] [Google Scholar]
  • 60.Titov N. Status of computerized cognitive behavioural therapy for adults. Aust N Z J Psychiatry. 2007 Feb;41(2):95–114. doi: 10.1080/00048670601109873. [DOI] [PubMed] [Google Scholar]
  • 61.Brick JM, Kalton G. Handling missing data in survey research. Stat Methods Med Res. 1996 Sep;5(3):215–238. doi: 10.1177/096228029600500302. [DOI] [PubMed] [Google Scholar]
  • 62.Grol R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care. 2001 Aug;39(8 Suppl 2):II46–1154. doi: 10.1097/00005650-200108002-00003. [DOI] [PubMed] [Google Scholar]
  • 63.Gustafsson I, Hvelplund A, Hansen KW, Galatius S, Madsen M, Jensen JS, Tilsted H, Terkelsen CJ, Jensen LO, Jørgensen E, Madsen JK, Abildstrøm SZ. Underuse of an invasive strategy for patients with diabetes with acute coronary syndrome: a nationwide study. Open Heart. 2015;2(1):e000165. doi: 10.1136/openhrt-2014-000165. https://openheart.bmj.com/cgi/pmidlookup?view=long&pmid=25685362. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Hulscher ME, Grol RP, van der Meer JW. Antibiotic prescribing in hospitals: a social and behavioural scientific approach. Lancet Infect Dis. 2010 Mar;10(3):167–175. doi: 10.1016/S1473-3099(10)70027-X. [DOI] [PubMed] [Google Scholar]
  • 65.McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA. The quality of health care delivered to adults in the United States. N Engl J Med. 2003 Jun 26;348(26):2635–2645. doi: 10.1056/NEJMsa022615. [DOI] [PubMed] [Google Scholar]
  • 66.IJzerman RVH, van der Vaart R, Evers AWM. Internet-based cognitive behavioral therapy among psychologists in a medical setting: a survey on implementation. J Med Internet Res. 2019 Aug 09;21(8):e13432. doi: 10.2196/13432. https://www.jmir.org/2019/8/e13432/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Richards DA, Ekers D, McMillan D, Taylor RS, Byford S, Warren FC, Barrett B, Farrand PA, Gilbody S, Kuyken W, O'Mahen H, Watkins ER, Wright KA, Hollon SD, Reed N, Rhodes S, Fletcher E, Finning K. Cost and Outcome of Behavioural Activation versus Cognitive Behavioural Therapy for Depression (COBRA): a randomised, controlled, non-inferiority trial. Lancet. 2016 Aug 27;388(10047):871–880. doi: 10.1016/S0140-6736(16)31140-0. https://linkinghub.elsevier.com/retrieve/pii/S0140-6736(16)31140-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Grilli R, Lomas J. Evaluating the message: the relationship between compliance rate and the subject of a practice guideline. Med Care. 1994 Mar;32(3):202–213. doi: 10.1097/00005650-199403000-00002. [DOI] [PubMed] [Google Scholar]
  • 69.Rogers E. Diffusion of Innovations. London: Free Press; 2003. [Google Scholar]
  • 70.Denis J, Hébert Y, Langley A, Lozeau D, Trottier L. Explaining diffusion patterns for complex health care innovations. Health Care Manage Rev. 2002;27(3):60–73. doi: 10.1097/00004010-200207000-00007. [DOI] [PubMed] [Google Scholar]
  • 71.Schreiweis B, Pobiruchin M, Strotbaum V, Suleder J, Wiesner M, Bergh B. Barriers and facilitators to the implementation of ehealth services: systematic literature analysis. J Med Internet Res. 2019 Nov 22;21(11):e14197. doi: 10.2196/14197. https://www.jmir.org/2019/11/e14197/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Kok G, Gottlieb NH, Peters GY, Mullen PD, Parcel GS, Ruiter RAC, Fernández ME, Markham C, Bartholomew LK. A taxonomy of behaviour change methods: an Intervention Mapping approach. Health Psychol Rev. 2015 Oct 15;:1–16. doi: 10.1080/17437199.2015.1077155. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia Appendix 1

English translation of invitation letter used to recruit respondents.

jmir_v22i8e18033_app1.docx (78.6KB, docx)
Multimedia Appendix 2

Survey development.

jmir_v22i8e18033_app2.docx (29KB, docx)
Multimedia Appendix 3

English translation of the online questionnaire as it was presented to the respondents.

jmir_v22i8e18033_app3.docx (27.9KB, docx)
Multimedia Appendix 4

Characteristics of the study sample and organizational characteristics.

jmir_v22i8e18033_app4.docx (25.6KB, docx)

Articles from Journal of Medical Internet Research are provided here courtesy of JMIR Publications Inc.

RESOURCES