Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Jan 1.
Published in final edited form as: Behav Ther. 2023 Jun 29;55(1):201–211. doi: 10.1016/j.beth.2023.06.003

Availability of internet-based cognitive-behavioral therapies (iCBTs) for depression: A systematic review

John F Buss a, Joshua S Steinberg b, Gregory Banks a, Deena Horani a, Lauren A Rutter a, Akash R Wasil c, Israel Ramirez a, Lorenzo Lorenzo-Luaces a
PMCID: PMC10787155  NIHMSID: NIHMS1931096  PMID: 38216233

Abstract

We examined the availability and components of internet-based cognitive-behavioral therapies (iCBTs) for depression tested in randomized-controlled trials (RCTs). The objectives of this literature review were to determine the extent to which research-validated iCBTs were available to the public, as well as to determine their therapeutic content. A literature review of randomized controlled trials for iCBTs was conducted on July 30th, 2021. In each iCBT search, interventions were disaggregated into treatment modules, comporting to prior work and compared to commercially available smartphone apps. Our search yielded 80 studies using 41 unique iCBTs. Of these, only six (15%) were completely available to the public, more than half were not publicly available (46%), and the remaining 39% were available to the public with some restrictions (e.g., those based on the user’s geographical location). When comparing iCBTs evaluated in RCTs to commercially available smartphone apps, we found that iCBTs were more likely to contain psychoeducation, cognitive restructuring, behavioral activation, problem-solving, and interpersonal communication components. iCBTs from RCTs contain evidence-based content but few are available to the public. Extending beyond efficacy, attention should be paid to the dissemination of iCBTs.

Keywords: Depression, cognitive-behavioral therapy (CBT), internet-based CBT (iCBT), mental health apps

Background

Depression is a leading contributor to the global burden of disability (Ferrari et al., 2013) with prospective epidemiological surveys suggesting that anywhere from 37–51% of individuals will meet criteria for a major depressive episode (MDE) by the age of 30 (Lorenzo-Luaces et al., 2015; Rohde et al., 2012). Although there are efficacious treatments for MDEs, including antidepressant medications and cognitive-behavioral therapies (CBTs), there are substantial barriers to treatment including financial costs (Dewa & Hoch, 2015; Goldberg & Steury, 2001) scarcity of available providers, and stigma associated with professional help-seeking (Andrade et al., 2014; Mojtabai et al., 2002).

Progress in the study and development of self-help interventions, particularly digital mental health interventions (DMHIs), suggests that scalable programs and materials may help reduce barriers to depression treatment (Karyotaki et al., 2018; Karyotaki et al., 2017). Indeed, many individuals already report a strong preference for using self-help approaches (Hanson et al., 2016), and clinicians have shown interest in incorporating low-intensity DMHIs such as iCBTs for patients on waiting lists (Peipert et al., 2022). The potential of DMHIs to improve upon face-to-face psychotherapy outcomes is also apparent (Lindhiem et al., 2015). However, there is substantial variation in the evidence base for these self-help programs. One way to establish efficacy of a treatment is to evaluate it in a randomized control trial (RCT). Meta-analyses of RCTs suggest that online and smartphone-based interventions can be effective for the treatment of depression (Ebert et al., 2015; Firth et al., 2017; Lattie et al., 2019; Lecomte et al., 2020). Although the use of the internet to reduce the burden of untreated depression appears promising, there are several reasons to be cautious about its application. For example, a recent review of publicly available mental health apps (MH apps) found that certain evidence-based treatment elements were rarely included in MH apps (Wasil et al., 2019). Here, MH apps were defined as any telephone applications, one can download from a commercial marketplace like the Apple App store or Google Play store. Of the popular MH apps Wasil et al. (2019) studied, where popularity was defined by the total number of downloads, few contained cognitive restructuring (31%), behavioral activation (31%), and none contained problem-solving (0%), even though these are common elements of evidence-based treatments for depression (Chorpita & Daleiden, 2009). Moreover, many of the commercially available apps lack a clear evidence base. A recent study that systematically searched the Google Play and Apple App Store suggested that only 3% of the reviewed 74 MH apps were supported by any peer-reviewed evidence (Larsen et al., 2019).

Of the evidence-based psychotherapies for depression, CBT is one of the most widely tested (Cuijpers et al., 2013). Consequently, CBT is the treatment modality that has been most frequently studied over the internet. Numerous meta-analyses have shown that internet-based CBTs (iCBTs) are efficacious treatments for depression (Andersson et al., 2014; Carlbring et al., 2018), though they appear most effective when delivered with some degree of human support (Karyotaki et al., 2018; Karyotaki et al., 2017). Evidence suggests iCBTs are roughly equivalent to face-to-face CBT (Carlbring et al., 2018) and are probably more cost-effective treatments (Nordgren et al., 2014). In addition to being efficacious in clinical trials, iCBTs have demonstrated effectiveness in analyses of routine care clinics from Sweden, Denmark, Norway, Canada, and Australia (Andersson & Hedman, 2013; Hedman et al., 2014; Titov et al., 2018). These analyses provide support for the adoption of iCBT interventions into clinical care. Practicing clinicians may be willing to use iCBTs to support their practice by assigning them to waitlist patients or by using them as adjunct material for current treatments. A recent meta-analysis on DMHIs indicated that there is little data on the adoption and uptake of RCT- tested DMHIs, due to a lack of reporting on these outcomes (Lattie et al., 2019). It is unknown to what extent empirically validated iCBTs have been disseminated and made available to the public.

To provide practical information regarding the extent to which iCBTs that have been studied in RCTs are available to the public, we conducted a systematic review of iCBT studies for depression and explored whether these iCBTs were available. We sought to identify whether 1.) evidence-based DMHIs like iCBTs are available but are not receiving traction due to the saturated DMHI market or 2.) evidence-based DMHIs like iCBTs are simply not publicly available. We describe the content within the iCBTs and assess whether they differ from popular MH apps. We also summarize the iCBTs that are available to the public; see Appendix. In doing so, we hope to present information that may be clinically meaningful and highlight the research-practice gap.

Methods

Search strategy

We obtained RCTs exploring the efficacy of iCBT for depression in adults by conducting a systematic review of the English literature up to July 30th, 2021. We combined terms relating to the internet, treatment (e.g., “psychotherapy”), and depression (e.g., “depression”, “major depressive disorder”) to find articles in PubMed, EMBASE, PsycINFO, and the Cochrane Library (see Appendix A for search terms). We also explored the possibility that out search missed other articles from prior systematic reviews and meta-analyses (e.g., Karyotaki et al., 2017, 2018) and included these articles as necessary.

Study selection criteria

To be eligible for our review, an article had to (a) recruit adults >18 years of age, (b) with some indication of depression (i.e., a diagnosis of major depressive disorder, with elevated symptoms of depression on a validated symptom questionnaire, or who were undergoing treatment for depression), (c) who were randomized to a treatment or a control group, and (d) the intervention was delivered only over the internet or phone application.

Articles in which a computer program was made available to participants via a physical copy (e.g., CD-ROM or flash drive; e.g., Sandoval et al., 2017), were not included, nor were articles in which the iCBT was delivered with other treatment components (I.e., adjunct therapy with a therapist or treatment coach was not allowed, however guided treatment coaching promoting adherence was allowed). We only included articles (e) in which treatment was compared to an inactive control condition (e.g., treatment as usual, waiting list, psychoeducation, or a placebo intervention) to establish assay sensitivity, or the ability to distinguish active from inactive treatments (Klein, 2000). Additionally, we (f) included interventions based on the principles of CBT, including behavioral activation and problem solving, as well as third-wave therapies (e.g., ACT, MBCT). If the literature explicitly stated the intervention was psychodynamic, motivational interviewing, or some other other non-CBT intervention, or if the preponderance of the content focused on non-CBT modalities, the study was not included. We included articles in which patients were treated (g) in the acute phase of depression (i.e., not prevention or relapse-prevention trials). Articles were also excluded if (h) the treatments was only focused on comorbid presentations (e.g., depression and alcohol use disorder). Treatments could have patients presenting with comorbid diagnoses; however, the articles were excluded if the iCBT treatment was focused on treatment of the comorbid diagnoses. Trials were also excluded if (i) treatment focused on a single symptom (e.g., anhedonia). These criteria were established to ensure the studies were treating depression as the primary diagnosis.

The total number of articles identified through the initial searches was 7,514. After duplicates were deleted, three of the authors (JB, GB, JS) examined the abstracts of 5,408 articles and screened them using the inclusion criteria. After this initial screening, three of the authors read the text of 527 articles, which produced 80 articles to be included in this systematic review. See Figure 1 for a PRISMA Flow Diagram. Discrepancies between the three researchers were resolved via consensus. In cases where there was conflicting data, the senior author (LL-L) made the final decision.

Figure 1:

Figure 1:

Open in a new tab

PRISMA flow diagram for randomized controlled trials using iCBT interventions for depression

Rating study availability

Our aim was to identify whether an iCBT was publicly available at the time of the search. We rated the availability of iCBTs using five categories: 1) not publicly available, 2) publicly available (i.e., accessible via the internet regardless of location or citizenship, and thus not including through one’s employer, insurance company, or government-based healthcare program), 3) conditionally available with geographical restrictions (e.g., only available to Australian residents, or through a geographically restricted healthcare program), 4) conditionally available with a registration code (i.e., registration code given by a research study or employee-assistance program), and 5) a combination of the prior four descriptions (e.g., available to everyone in one geographic area but also available with a registration code). We acknowledge that the current definition of ‘publicly available’ sets a high standard in that we only included programs that were accessible via the internet regardless of location (iI.e., internationally with internet access), and without needing a specific registration code. This definition of fully publicly available does not include iCBTs that are accessible in specific country populations via a government program or healthcare system (e.g., Deprexis). To determine availability, we searched for the iCBT package online via: 1) the Google search engine, 2) the Apple App Store, 3) the Google Play Store, and 4) using descriptions provided within the article (e.g., if the authors provided a specific link to the iCBT). If we could not find information on the iCBT (e.g., the name was not present in the article), or we deemed the intervention conditionally available we contacted the corresponding authors of the respective study to confirm whether the iCBTs were publicly available. We attempted to be sensitive to possible differences in the way iCBT programs were named in the literature vs. commercially. For example, the depression iCBT package “This Way Up” online course is sometimes referred to as “The Sadness Program” in the literature (Titov et al., 2010).

Rating study content

Some iCBTs were used throughout different studies in their original versions, while others were adapted versions of previously-studied iCBTs (e.g., for specific populations, across different languages). The types of adaptations ranged, from slight cultural and language modifications to manipulations of full content modules. Information from the publication, protocol paper, and trial registration were used to determine if the iCBT was adapted from another iCBT intervention.

To understand the therapeutic content of the iCBTs, we coded for 14 different treatment components recognized in prior meta-analyses (López-López et al., 2019; Pompoli et al., 2018) and commonly found in treatment manuals. The research team identified the content of each intervention by a.) looking into the intervention itself, if available to the public or behind a paywall, b.) reading the main article description of the intervention, c.) reading a trial protocol detailing the intervention’s components, d.) reading the trial registration for any additional information. While we were informed by the specific labels that researchers used to define the therapeutic content (e.g., if an iCBT was coded as having “cognitive restructuring” it would be highly likely to have cognitive restructuring), we investigated the content which was sometimes introducing using more colloquial terms (e.g., “cognitive restructuring” could be referred to as “check your thoughts”). Each article was rated by two of the study authors. JB coded all the studies and DH and JS served as second raters for the other articles. Overall, there was moderate agreement across categories (K= 0.56). The senior author (LL-L) resolved disagreements. The 14 modules were: (1) psychoeducation (i.e., education about the theoretical groundwork of depression and its treatment; K= 0.55), (2) cognitive restructuring (i.e., attempts to challenge or change negative thoughts or rumination; K= 0.72), (3) behavioral activation (i.e., attempts to increase positive activities or decrease avoidance; K= 0.54), (4) problem solving (i.e., developing skills for identifying, selecting, and solving problems; K= 0.78), (5) interpersonal communication (e.g., assertiveness training, communication skills; K= 0.54), (6) positive psychology (e.g., elements of positive emotions, including gratitude, hope, savoring, forgiveness, altruism; K= 0.67), (7) emotion-focused (i.e., emotional awareness, emotional regulation; K= 0.47), (8) relaxation (e.g., deep breathing, progressive muscle relaxation; K= 0.52), (9) mindfulness (e.g., meditation, accepting painful sensations; K= 0.63), (10) expressive writing (e.g., writing as self-reflection; K= 0.52), (11) sleep hygiene (e.g., K= 0.57), (12) lifestyle (e.g., diet, exercise, K= 0.58), (13) anxiety (e.g., anxiety-specific psychoeducation, elements of exposure therapy; K= 0.28), (14) relapse prevention (e.g., identification of triggers, teaching the patient how to be “their own therapist”; K= 0.45).

When interventions were publicly available, we accessed them to verify their content. When not, we accepted the definitions provided in research articles, public media, or commercial content. In other words, if an application purported to use a cognitive restructuring module, we assumed it was accurately portraying itself. Our classifications were relatively broad. For example, if one treatment element was described (i.e., gratitude), we coded the iCBT as containing that treatment component (e.g., positive psychology). If a component was not mentioned in the trial registry, protocol, original publication, or intervention, it was assumed the treatment did not contain that component.

Comparison of iCBTs and MH Apps

To establish a benchmark against which to evaluate the presence of evidence-based content within iCBTs, we compared the frequency with which different components were used in iCBT vs. the frequency with which the same content could be found in popular smartphone apps for depression using chi-square tests. To do this, we drew upon the popular MH apps for depression identified by Wasil et al. (2019) and colleagues. For 6/14 of our categories, the coding used by Wasil et al. overlapped with ours (i.e., psychoeducation, cognitive restructuring, problem-solving, relaxation, mindfulness, and emotion-focused). For four of our categories, the coding used by Wasil et al. was related to our categories, but was more granular so we grouped these related interventions (i.e., anxiety content = exposure or stimulus control; positive psychology = expressing kindness to self or others; behavioral activation = activity scheduling, skill-building/behavioral rehearsal, or behavioral contracting; interpersonal communication = communication skills, communication analysis, or assertiveness training). For the remaining categories, there was no overlap between our categorization and Wasil et al. 2019 (i.e., sleep hygiene, lifestyle, expressive writing, and relapse prevention), thus these four categories were not included in the overall comparison. In addition to comparing the identified iCBTs to the most popular MH apps the research team compared the frequency in treatment components between the publicly available iCBTs and the iCBTs that were unavailable to the public using chi-square tests. Finally, we compared the average total number of evidence-based treatment modules in iCBTs to the MH apps identified by Wasil et al. 2019. The team used a one-sample t-test to determine significance.

Risk of Bias Assessment

Risk of bias was assessed by two authors (JB, JS) independently using the Cochrane risk of bias tool for randomized trials (Higgins et al., 2011). The risk of bias tool assesses bias on the following domains: selection bias, performance bias, attrition bias, reporting bias, and detection bias using 22 criteria. For every criterion, risk of bias was assessed as (1) low risk of bias, (2) some concerns of bias, and (3) high risk of bias. Conflicts of opinion were discussed with the senior author (LL-L) until consensus was reached. For assessment of bias, we used the iCBT article identified, available pre-registrations, and study protocol papers if they were available.

Results

Our search identified 80 randomized controlled trials (see Figure 1). These trials used 63 iCBTs. However, of the iCBTs we identified, 22 were adaptations (e.g., GET.ON MOOD ENHANCER FOR DIABETES). Thus, focusing only on the “original” programs, we identified 41 unique iCBT treatments.

Availability

Of the 41 programs identified, six were available without any restrictions at the time of drafting this manuscript (See Appendix). The iCBTs that were fully accessible were: Beating the Blues, MoodGYM, E-Couch, SuperBetter - iCBT version, Thought Challenger, and Moodkit. Of the remaining 35 that were not accessible, six were only available in specific countries or regions, as verified by postal code or phone number (i.e., Deprexis, Alles Onder Controle, MumMoodBooster, Coping with Depression, Managing Your Mood Course, and Spirits Healing). Six required a specific registration link (i.e., GET.ON Mood Enhancer, Space from Depression, Mom-Net, Moodivate, iFightDepression, Meru Health Program). Two iCBTs (i.e., Smiling is Fun and Emyna) were both geographically restricted and required a registration link. One iCBT (i.e., Thrive) was available for bulk purchases only, being marketed to corporations. Lastly, one iCBT, “This Way Up”, also known in the literature as “The Sadness Program”, was available to practicing clinicians regardless of location. However, the purely self-guided version was only available in Australia. The remaining 19 iCBTs (45%) were completely unavailable to the public at the time of our search. Of the 22 adaptations we found, we could not identify a single one that was made publicly available.

Our search also found differences in the number of studies supporting each iCBT. The most frequently researched iCBT for depression was “Deprexis”, which appeared in nine of the 80 RCTs. The program has been so widely studied that it has its own meta-analyses, which documents the intervention’s efficacy (Twomey et al., 2020). Deprexis has been translated into nine languages and has been heavily researched, but Deprexis is only available in five select Western countries (i.e., France, Germany, United Kingdom, Switzerland, and the United States), and is also behind a pay barrier of $399. Another frequently researched iCBT for depression, “This Way Up” (The Sadness Program), has been tested in at least four RCTs and once as an adaptation, yet the self-guided program is only available in Australia.

Content

Among the iCBTs, most programs included psychoeducation (n = 33, 80%), cognitive restructuring (n = 30, 73%), and behavioral activation (n = 30, 73%). Despite our search for depression treatments, 8 (20%) of the interventions included an anxiety component. The least frequent treatment components were lifestyle (5; 12%), expressive writing (5; 12%), and emotion-focused (5; 12%). See Table 1, below for content module frequencies.

Table 1: –

Comparison of content frequency in iCBTs for depression from randomized controlled trials vs. popular mental health apps (see Wasil et al., 2019)

Treatment modules iCBTs (n=41) Popular apps(n=16) P-valuea
Psychoeducation 33 (80%) 8 (50%) 0.046*
Relaxation 10 (24%) 6 (38%) 0.34
Mindfulness 11 (27%) 6 (38%) 0.52
Cognitive Restructuring 30 (73%) 5 (31%) 0.006*
Behavioral Activation 30 (73%) 5 (31%) 0.006*
Emotion-Focused 5 (12%) 4 (25%) 0.25
Problem Solving 13 (32%) 0 (0%) 0.01*
Interpersonal Communication 11 (27%) 0 (0%) 0.02*
Positive Psychology 7 (17%) 4 (25%) 0.48
Anxiety 8 (20%) 0 (0%) 0.09
Sleep Hygiene 10 (24%) N/A N/A
Relapse Prevention 9 (22%) N/A N/A
Lifestyle 5 (12%) N/A N/A
Expressive Writing 5 (12%) N/A N/A
Open in a new tab
a

= P-value for Chi-Square test. N/A = treatment components were not represented in Wasil et al.’s code.

*

Statistically significant at the α = 0.05 level.

Benchmarking analysis

When comparing the iCBT packages we identified to MH apps Wasil et al. (2019) identified, several patterns emerged (see Table 1). The iCBTs were more likely to contain psychoeducation, cognitive restructuring, behavioral activation, problem-solving, and interpersonal communication. When comparing the six publicly available iCBTs to the identified popular MH apps, the publicly available apps were more likely to have cognitive restructuring (p = 0.012) and problem solving (p = 0.013; see Appendix). Additionally, the team found that there were differences in the number of empirically-supported treatment components for both iCBTs and MH apps identified by Wasil et al., where iCBTs (Mean = 4.6 modules, SD = 2.3) were more likely to have more empirically-supported treatments than MH apps (Mean = 2.4 modules; t = 6.01, df = 40, p < 0.0001).

We explored whether the content of the publicly available iCBTs (n = 6) differed from the content of the iCBTs that were inaccessible (n = 35). There were no statistically significant differences in the likelihood of containing specific treatment elements at P ≤ 0.05. See Table 2.

Table 2: –

Comparison of content frequency in iCBTs for depression publicly available research-validated iCBTs vs. unavailable research-validated iCBTs

Treatment modules Unavailable iCBTs (n=35) Available iCBTs (n=6) P-valuea
Psychoeducation 29 (85%) 4 (67%) 0.58
Relaxation 8 (24%) 2 (33%) 0.62
Mindfulness 10 (29%) 0 (0%) 0.31
Cognitive Restructuring 25 (74%) 6 (100%) 0.31
Behavioral Activation 26 (76%) 5 (83%) 1.0
Emotion-Focused 3 (9%) 1 (17%) 0.48
Problem Solving 11 (32%) 3 (50%) 0.39
Interpersonal Communication 2 (33%) 0 (0%) 1.0
Positive Psychology 1 (17%) 6 (17%) 1.0
Anxiety 7 (21%) 1 (17%) 1.0
Sleep Hygiene 10 (29%) 1 (17%) 1.0
Relapse Prevention 10 (29%) 0 (0%) 0.31
Lifestyle 3 (9%) 1 (17%) 0.48
Expressive Writing 3 (9%) 1 (17%) 0.48
Open in a new tab
a

= P-value for Fisher exact test.

*

Statistically significant at the α = 0.05 level.

Risk of Bias

Risk of bias information is available in the Appendix and suggests that 87.8% studies had at least some risk of bias. However, most of this bias was owing to the use of self-report as an outcome measure. On the other domains, studies appeared to be mostly at low risk ranging from 72% (selection of the reported result)-95.1% (deviation from intended interventions). See Figure 2 in Appendix for a summary.

Discussion

We systematically reviewed the availability and content of iCBTs studied in RCTs. Our review had several aims. First, we evaluated the extent to which evidence-based iCBTs are available to potential patients and clinicians. Our search yielded 80 RCTs. Excluding adaptations, we were able to identify 41 distinct iCBTs. Of those studies, only a tiny fraction (15%) were publicly available with no restrictions. Secondly, we sought to identify the most frequently utilized components in iCBTs. We benchmarked these frequencies against a published review of MH apps (Wasil et al., 2019). Relative to popular MH apps identified by Wasil et al. 2019, iCBTs more frequently endorsed treatment modules, including psychoeducation, cognitive restructuring, problem-solving, and behavioral activation. Our results suggest iCBTs tested in the context of RCTs differ from popular MH apps, such that iCBTs are more likely to have evidence-based components, but they are predominantly not available to the public (Wasil et al., 2019).

Before interpreting our findings, it is worth considering several limitations. First, only published articles in the English language were included. Second, our search is limited to iCBTs for adults; therefore, our search is not representative of programs for youth. Third, the small number of publicly available iCBTs identified in our search potentially reduced the power to detect differences in component frequencies. Fourth, we explored whether iCBTs were publicly available by using popular search methods and contacting study authors. It is possible that this search method does not fully emulate how a prospective patient would search for online help. Additionally, the current definition of publicly available purposely sets a high standard. It does not include iCBTs available in countries through universal healthcare programs or international employers, so these accessibility disparities could be highlighted. If the current review were to adopt a significantly less strict definition of publicly available (i.e., available to clients not on a research trial), our results would indicate that many of these iCBTs are “publicly available” or at least distributed to specific populations. Fifth, the purpose of this analysis was to determine whether previously research-validated studies were disseminated to the public. The research team did not explicitly look at the research base of the most popular mental health apps, though we used a dataset that drew from these more widely disseminated interventions (Wasil et al., 2019). While it’s important to identifying barriers towards disseminating efficacious iCBTs to the public, growing the research base of these most popular used mental health apps is also an important strategy to consider (Wasil et al., 2020). Sixth, we note that the comparisons made between MH apps and iCBT found in the literature are not necessarily directly comparable as our search was limited to CBT apps, while the most populat app store apps were not limited by content. Thus, the results of these comparisons are preliminary and warrant further investigation.

Although our methods are not without limitations, existing data suggests that people indeed use popular search engines to engage in help-seeking behaviors (Fiksdal et al., 2014). Finally, it is possible that we may have made a mistake in the search process. Despite some limitations in our analysis, our paper fills in several gaps in the extant literature. To our knowledge, there has been no study examining the extent to which iCBTs that have been studied in academic work have been made available to the public. The lack of evidence-based content in popular MH apps could be the product of a large number of apps that are not based on research, research- based interventions like iCBTs not being well-disseminated, or some combination of both of these factors. Our results suggest that the poor dissemination of iCBT could contribute to the lack of evidence-based content in popular MH apps. Thus, these findings provide clinicians and health care providers with essential data for making informed, evidence-based decisions when choosing an internet-based treatment. Additionally, this article is the first to note the research-treatment gap between the publicly available and unavailable iCBTs for the treatment of depression. Finally, we were able to benchmark the frequency with which the content of iCBTs differed from a prior publication.

While we reviewed iCBTs studied in RCTs, other data sources evaluate publicly available internet-based interventions. One non-profit project, Psyberguide.org, provides directions for navigating the MH app marketplace. The project primarily categorizes and disseminates information on MH apps and online interventions based on their credibility, user experience, and data transparency. It has been shown that there are more than 10,000 MH apps (Torous et al., 2019). Of these, just four (Headspace, Youper, Wysa, and Calm) accounted for roughly 90% of all MH app downloads (Wasil et al., 2020). Nonetheless, most (63%) of all apps identified for depression contain zero monthly users (Wasil et al., 2021). These data suggest that resources should be spent further evaluating apps that dominate the app market (Wasil et al., 2020). Additionally, it would behoove app developers and researchers to consider how novel MH apps would differentiate themselves, rather than further saturating the market and adding to the decision-making burden of those seeking resources.

As expected, given their central role in cognitive-behavioral therapy, psychoeducation, behavioral activation, and cognitive restructuring were the most frequently utilized components in the identified iCBTs. Of note, these treatment components are not well-represented in MH apps. Additionally, the popular MH apps appeared to have fewer evidence-based treatment components, than the developed iCBTs. Whether shorter treatment packages are preferred by consumers or industry has not been thoroughly explored. First, it is possible that the CBT community has not thoroughly attempted disseminating CBT content into online domains. Second, it is possible that consumers are not as interested in the topics of traditional iCBTs such as cognitive restructuring as in other components of “third-wave” CBT. Finally, certain elements such as relaxation and mindfulness may appeal more to private industry (Farias & Wikholm, 2016). This lack of iCBT dissemination may not be through a lack of effort. Among the authors contacted, many cited a lack of resources (e.g., funding, personnel) for discontinuing their iCBT. More research should look into investigating these barriers to iCBT implementation.

Unexpectedly, relapse prevention was included in only 24% of iCBTs for depression. The lack of relapse prevention in these iCBTs is concerning, considering high relapse rates in some subgroups of depressed patients (Lorenzo-Luaces et al., 2015). In addition, relapse prevention is an essential component of CBT and is often reported to be a part of the therapy’s prophylactic effects. Thus it is highly recommended relapse prevention components be included in future iCBTs. Additionally, clinicians administering these iCBTs should be aware of the absence of relapse prevention content and provide supplemental material to patients to prevent relapse when appropriate.

We hope this study serves as a launching pad for future studies regarding the dissemination of iCBTs. The current literature suggests that while many iCBTs are studied, very few are well-disseminated. Some of the authors attributed the failure to disseminate iCBTs to a lack of resources, though future research should explore these reasons more systematically, than we have done here. Although creating and maintaining iCBTs may require a great deal of skill and resources, one cost-effective alternative may be to share the intervention content on open repositories. For example, researchers often share materials on the Open Science Framework (OSF). While the site itself may not allow someone to host an iCBT per se (i.e., via an app or webpage), researchers may share intervention content for future research. Second, A recent overview of recommendations for improving access to DMHIs highlighted multiple items that researchers could use when creating new DMHIs for the market (Lattie et al., 2022). Our findings support calls for funding agencies to establish links between researchers and industry partners. Indeed, the National Institute of Health’s (NIH) recent R01 grant for funding the dissemination and effectiveness of DMHIs is one step in the right direction (NIH, 2022), as well as the usage of the Small Business Technology Transfer (STTR) Grant R41 and R42. However, few funding opportunities have been dedicated to sustaining and disseminating DMHIs, especially in collaboration with industry partners. Finally, these interventions should be able to reach a more comprehensive array of diverse populations. There needs to be more work in the digital mental health field identifying for which populations iCBTs are being adopted and whom they are effective for, as oppressed minority groups with the highest need could potentially be further barred from these treatments due to their untailored nature.

Conclusion

Internet mental health interventions have great potential to overcome a multitude of treatment barriers for the general public. However, a significant research-practice gap between popular MH apps and empirically validated iCBTs for depression exists. The content in iCBTs significantly differed from popular MH apps, with iCBTs more likely to contain psychoeducation, cognitive restructuring, behavioral activation, problem-solving, and interpersonal communication components; however further research should be done to validate these preliminary comparisons. The efficacy of guided and unguided iCBTs has been well-established. What is now needed is the dissemination of these iCBTs and more studies of them in naturalistic contexts.

Supplementary Material

1

Highlights.

  • Most RCT-tested iCBTs are not accessible to the public

  • A significant number of RCT-tested iCBTs are geographically restricted

  • iCBTs are more likely to contain evidence-based components, than popular MH apps

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of interest

We report no conflict of interest relevant to the current publication.

Data Availability Statement

The data that support the findings of this study are available upon request in (NIMH, 2022).

References

  1. Andersson G, Cuijpers P, Carlbring P, Riper H, & Hedman E (2014). Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: A systematic review and meta-analysis. World Psychiatry, 13 (3), 288–295. 10.1002/wps.20151 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Andersson G, & Hedman E (2013). Effectiveness of guided internet-based cognitive behavior therapy in regular clinical settings. Verhaltenstherapie, 23 (3), 140–148. 10.1159/000354779 [DOI] [Google Scholar]
  3. Andrade LH, Alonso J, Mneimneh Z, Wells J, Al-Hamzawi A, Borges G, Bromet E, Bruffaerts R, De Girolamo G, De Graaf R, et al. (2014). Barriers to mental health treatment: Results from the who world mental health surveys. Psychological Medicine, 44 (6), 1303–1317. 10.1017/S0033291713001943 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Carlbring P, Andersson G, Cuijpers P, Riper H, & Hedman-Lagerlöf E (2018). Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: An updated systematic review and meta-analysis. Cognitive Behaviour Therapy, 47 (1), 1–18. 10.1080/16506073.2017.1401115 [DOI] [PubMed] [Google Scholar]
  5. Chorpita BF, & Daleiden EL (2009). Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology, 77 (3), 566. 10.1037/a0014565 [DOI] [PubMed] [Google Scholar]
  6. Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, & Dobson KS (2013). A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. The Canadian Journal of Psychiatry, 58 (7), 376–385. 10.1177/070674371305800702 [DOI] [PubMed] [Google Scholar]
  7. Dewa CS, & Hoch JS (2015). Barriers to mental health service use among workers with depression and work productivity. Journal of Occupational and Environmental Medicine, 57 (7), 726. 10.1097/JOM.0000000000000472 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Ebert DD, Zarski A-C, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, & Riper H (2015). Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: A meta-analysis of randomized controlled outcome trials. PloS One, 10 (3), e0119895. 10.1371/journal.pone.0119895 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Farias M, & Wikholm C (2016). Has the science of mindfulness lost its mind? BJPsych bulletin, 40 (6), 329–332. 10.1192/pb.bp.116.053686 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Ferrari AJ, Charlson FJ, Norman RE, Patten SB, Freedman G, Murray CJ, Vos T, & Whiteford HA (2013). Burden of depressive disorders by country, sex, age, and year: Findings from the global burden of disease study 2010. PLoS Medicine, 10 (11). 10.1371/journal.pmed.1001547 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Fiksdal AS, Kumbamu A, Jadhav AS, Cocos C, Nelsen LA, Pathak J, & McCormick JB (2014). Evaluating the process of online health information searching: A qualitative approach to exploring consumer perspectives. Journal of Medical Internet Research, 16 (10), e3341. 10.2196/jmir.3341 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, & Sarris J (2017). The efficacy of smartphone-based mental health interventions for depressive symptoms: A meta-analysis of randomized controlled trials. World Psychiatry, 16 (3), 287–298. 10.1002/wps.20472 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Goldberg RJ, & Steury S (2001). Depression in the workplace: Costs and barriers to treatment. Psychiatric Services, 52 (12), 1639–1643. 10.1176/appi.ps.52.12.1639 [DOI] [PubMed] [Google Scholar]
  14. Hanson K, Webb TL, Sheeran P, & Turpin G (2016). Attitudes and preferences towards self-help treatments for depression in comparison to psychotherapy and antidepressant medication. Behavioural and Cognitive Psychotherapy, 44 (2), 129–139. 10.1017/S1352465815000041 [DOI] [PubMed] [Google Scholar]
  15. Hedman E, Ljótsson B, Kaldo V, Hesser H, El Alaoui S, Kraepelien M, Andersson E, Rück C, Svanborg C, Andersson G, et al. (2014). Effectiveness of internet-based cognitive behaviour therapy for depression in routine psychiatric care. Journal of Affective Disorders, 155, 49–58. 10.1016/j.jad.2013.10.023 [DOI] [PubMed] [Google Scholar]
  16. Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Savović J, Schulz KF, Weeks L, & Sterne JA (2011). The cochrane collaboration’s tool for assessing risk of bias in randomised trials. BMJ, 343. 10.1136/bmj.d5928 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Karyotaki E, Kemmeren L, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, Mackinnon A, Meyer B, Botella C, et al. (2018). Is self-guided internet-based cognitive behavioural therapy (icbt) harmful? an individual participant data meta-analysis. Psychological Medicine, 48 (15), 2456–2466. 10.1017/S0033291718000648 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, Mackinnon A, Meyer B, Botella C, Littlewood E, et al. (2017). Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: A meta-analysis of individual participant data. JAMA Psychiatry, 74 (4), 351–359. 10.1001/jamapsychiatry.2017.0044 [DOI] [PubMed] [Google Scholar]
  19. Klein DF (2000). Flawed meta-analyses comparing psychotherapy with pharmacotherapy. American Journal of Psychiatry, 157 (8), 1204–1211. 10.1176/appi.ajp.157.8.1204 [DOI] [PubMed] [Google Scholar]
  20. Larsen ME, Huckvale K, Nicholas J, Torous J, Birrell L, Li E, & Reda B (2019). Using science to sell apps: Evaluation of mental health app store quality claims. NPJ Digital Medicine, 2 (1), 1–6. 10.1038/s41746-019-0093-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Lattie EG, Adkins EC, Winquist N, Stiles-Shields C, Wafford QE, & Graham AK (2019). Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: Systematic review. Journal of Medical Internet Research, 21 (7), e12869. 10.2196/12869 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Lattie EG, Stiles-Shields C, & Graham AK (2022). An overview of and recommendations for more accessible digital mental health services. Nature Reviews Psychology, 1 (2), 87–100. 10.1038/s44159-021-00003-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lecomte T, Potvin S, Corbière M, Guay S, Samson C, Cloutier B, Francoeur A, Pennou A, Khazaal Y, et al. (2020). Mobile apps for mental health issues: Meta-review of meta-analyses. JMIR mHealth and uHealth, 8(5), e17458. 10.2196/17458 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lindhiem O, Bennett CB, Rosen D, & Silk J (2015). Mobile technology boosts the effectiveness of psychotherapy and behavioral interventions: A meta-analysis. Behavior Modification, 39 (6), 785–804. 10.1177/0145445515595198 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. López-López JA, Davies SR, Caldwell DM, Churchill R, Peters TJ, Tallon D, Dawson S, Wu Q, Li J, Taylor A, et al. (2019). The process and delivery of cbt for depression in adults: A systematic review and network meta-analysis. Psychological Medicine, 49 (12), 1937–1947. 10.1017/S003329171900120X [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lorenzo-Luaces L, German RE, & DeRubeis RJ (2015). It’s complicated: The relation between cognitive change procedures, cognitive change, and symptom change in cognitive therapy for depression. Clinical Psychology Review, 41, 3–15. 10.1016/j.cpr.2014.12.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Mojtabai R, Olfson M, & Mechanic D (2002). Perceived need and help-seeking in adults with mood, anxiety, or substance use disorders. Archives of General Psychiatry, 59 (1), 77–84. 10.1001/archpsyc.59.1.77 [DOI] [PubMed] [Google Scholar]
  28. NIMH. (2022). National Institute of Mental Health Data Archive kernel description. Retrieved June 29, 2022, from https://nda.nih.gov/
  29. Nordgren LB, Hedman E, Etienne J, Bodin J, Kadowaki Å, Eriksson S, Lindkvist E, Andersson G, & Carlbring P (2014). Effectiveness and cost-effectiveness of individually tailored internet-delivered cognitive behavior therapy for anxiety disorders in a primary care population: A randomized controlled trial. Behaviour Research and Therapy, 59, 1–11. 10.1016/j.brat.2014.05.007 [DOI] [PubMed] [Google Scholar]
  30. Peipert A, Krendl AC, Lorenzo-Luaces L, et al. (2022). Waiting lists for psychotherapy and provider attitudes toward low-intensity treatments as potential interventions: Survey study. JMIR Formative Research, 6 (9), e39787. 10.2196/39787 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Pompoli A, Furukawa TA, Efthimiou O, Imai H, Tajika A, & Salanti G (2018). Dismantling cognitive-behaviour therapy for panic disorder: A systematic review and component network meta-analysis. Psychological Medicine, 48 (12), 1945–1953. 10.1017/S0033291717003919 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Rohde P, Stice E, & Gau JM (2012). Effects of three depression prevention interventions on risk for depressive disorder onset in the context of depression risk factors. Prevention Science, 13 (6), 584–593. 10.1007/s11121-012-0284-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Sandoval LR, Buckey JC, Ainslie R, Tombari M, Stone W, & Hegel MT (2017). Randomized controlled trial of a computerized interactive media-based problem solving treatment for depression. Behavior Therapy, 48 (3), 413–425. 10.1016/j.beth.2016.04.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, & Solley K (2010). Internet treatment for depression: A randomized controlled trial comparing clinician vs. technician assistance. PloS One, 5 (6), e10939. 10.1371/journal.pone.0010939 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Titov N, Dear B, Nielssen O, Staples L, Hadjistavropoulos H, Nugent M, Adlam K, Nordgreen T, Bruvik KH, Hovland A, et al. (2018). Icbt in routine care: A descriptive analysis of successful clinics in five countries. Internet Interventions, 13, 108–115. 10.1016/j.invent.2018.07.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Torous J, Andersson G, Bertagnoli A, Christensen H, Cuijpers P, Firth J, Haim A, Hsin H, Hollis C, Lewis S, et al. (2019). Towards a consensus around standards for smartphone apps and digital mental health. World Psychiatry, 18 (1), 97. 10.1002/wps.20592 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Twomey C, O’Reilly G, Bültmann O, & Meyer B (2020). Effectiveness of a tailored, integrative internet intervention (deprexis) for depression: Updated meta-analysis. PLoS One, 15 (1), e0228100. 10.1371/journal.pone.0228100 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Wasil AR, Gillespie S, Schell T, Lorenzo-Luaces L, & DeRubeis RJ (2021). Estimating the real-world usage of mobile apps for mental health: Development and application of two novel metrics. World Psychiatry, 20 (1), 137. 10.1002/wps.20827 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Wasil AR, Gillespie S, Shingleton R, Wilks CR, & Weisz JR (2020). Examining the reach of smartphone apps for depression and anxiety. American Journal of Psychiatry, 177 (5), 464–465. 10.1176/appi.ajp.2019.19090905 [DOI] [PubMed] [Google Scholar]
  40. Wasil AR, Venturo-Conerly KE, Shingleton RM, & Weisz JR (2019). A review of popular smartphone apps for depression and anxiety: Assessing the inclusion of evidence- based content. Behaviour Research and Therapy, 123, 103498. 10.1016/j.brat.2019.103498 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1931096-supplement-1.docx (38.5KB, docx)

Data Availability Statement

The data that support the findings of this study are available upon request in (NIMH, 2022).

RESOURCES