Skip to main content
Internet Interventions logoLink to Internet Interventions
. 2018 Apr 15;16:52–64. doi: 10.1016/j.invent.2018.04.002

Efficacy and cost-effectiveness of guided and unguided internet- and mobile-based indicated transdiagnostic prevention of depression and anxiety (ICare Prevent): A three-armed randomized controlled trial in four European countries

Kiona K Weisel a, Anna-Carlotta Zarski a,b, Thomas Berger c, Tobias Krieger c, Michael P Schaub d, Christian T Moser c, Matthias Berking a, Michelle Dey d, Cristina Botella e,f, Rosa Baños f,g, Rocio Herrero e,f, Ernestina Etchemendy f,h, Heleen Riper i,j, Pim Cuijpers i,j, Felix Bolinski i,j, Annet Kleiboer i,j, Dennis Görlich k, Jennifer Beecham l, Corinna Jacobi m, David D Ebert a,
PMCID: PMC6364519  PMID: 30775265

Abstract

Background

Depression and anxiety are highly prevalent and often co-occur. Several studies indicate the potential of disorder-specific psychological interventions for the prevention of each of these disorders. To treat comorbidity, transdiagnostic treatment concepts seem to be a promising approach, however, evidence for transdiagnostic concepts of prevention remains inconclusive. Internet- and mobile-based interventions (IMIs) may be an effective means to deliver psychological interventions on a large scale for the prevention of common mental disorders (CMDs) such as depression and anxiety. IMIs have been shown to be effective in treating CMDs, e.g. in reducing symptoms of depression and anxiety. However, there is a lack of studies examining the efficacy of interventions reducing the incidence of CMDs. Moreover, the comparative cost-effectiveness of guided versus unguided IMIs for the prevention of depression and anxiety has not been studied yet. Hence, this study aims at investigating the (cost-) effectiveness of guided and unguided internet- and mobile-based transdiagnostic individually tailored indicated prevention of depression and anxiety.

Methods

A multi-country three-armed randomized controlled trial will be conducted to compare a guided and unguided intervention to treatment as usual (TAU). Both active conditions are based on the same intervention, ICare Prevent, and differ only with regard to guidance format. Altogether, 954 individuals with subclinical symptoms of depression (CES-D ≥ 16) and anxiety (GAD-7 ≥ 5) who do not have a full-blown disorder will be recruited in Germany, Switzerland, Spain and the Netherlands, and randomized to one of three conditions (guided intervention, unguided intervention, or TAU). The TAU arm will receive access to the training after a 12-month waiting period. The primary outcome will be time to CMD onset (any depression/anxiety disorder) within a follow-up period of 12 months after baseline. Secondary outcomes will include disorder-specific symptom severity (depression/anxiety) assessed by diagnostic raters blinded to intervention condition at post-intervention, self-reports, acceptability, health related quality of life, and psychosocial variables associated with developing a CMD. Assessments will take place at baseline, mid-intervention (5 weeks into the intervention), post-intervention (8 weeks after randomization) and follow-up (6 and 12 months after randomization). Data will be analyzed on an intention-to-treat basis and per protocol. Cost-effectiveness will be evaluated from a public health and a societal perspective, including both direct and indirect costs.

Discussion

The present study will further enhance the evidence-base for transdiagnostic preventive interventions and provide valuable information about optimal trade-off between treatment outcome and costs.

Trial registration

German Clinical Trial Registration (DRKS - http://www.drks.de/drks_web/): DRKS00011099.

Keywords: Prevention, Transdiagnostic, Depression, Anxiety, Internet-based, Randomized controlled trial

Highlights

  • Depression and anxiety are highly comorbid, but often not treated together.

  • Transdiagnostic treatment provides potential to reduce comorbidity.

  • Individual tailoring can help adapt treatment to individual profiles.

  • Large multi-country transdiagnostic prevention trial in a subclinical population.

  • Cost-effectiveness for a guided and unguided internet- and mobile-based intervention will be investigated.

1. Introduction

Depression and anxiety disorders, due to their high prevalence, high disease burden, and common comorbidity (Brenes, 2007; Brown and Barlow, 1992; Kessler et al., 2005; Klein Hofmeijer-Sevink et al., 2012; Saarni et al., 2007; Wang et al., 2009), have been identified as two of the leading causes of disability (Baxter et al., 2014; Global Burden of Disease Study 2013 et al., 2015) causing an enormous economic burden through, e.g. reduced workforce participation, occupational impairment, and lost productivity (Berto et al., 2000; Birnbaum et al., 2010; Greenberg and Birnbaum, 2005; World Health Organization, 2008).

A variety of effective interventions for the treatment of common mental disorders (CMDs) have been developed over the past decades, and their efficacy has been demonstrated in a large number of randomized controlled trials (RCTs) (Cuijpers et al., 2008, Cuijpers et al., 2014b; Hofmann and Smits, 2008).

However, even though effective treatment modalities exist, most individuals in need of treatment for depression and anxiety remain untreated. What also adds to the unmet need for treatment, even when assuming the hypothetical scenario of 100% coverage and compliance to evidence-based treatments, is that only approximately one third of the disease burden attributable to MDD and half of the disease burden attributable to ADs could be averted with current treatments (Andrews et al., 2004; Chisholm et al., 2004). For this reason, attention is increasingly focusing on prevention of CMDs, by targeting individuals before the onset of a full syndrome CMD (Emmelkamp et al., 2014).

Emerging evidence indicates the potential of disorder-specific interventions to prevent the development of MDD (van Zoonen et al., 2014). However, evidence on the efficacy of interventions aiming to prevent the onset of anxiety especially long-term, is inconclusive. This is indicated by a meta-analysis showing that initial short-term effects diminish over time (Zalta, 2011). Moreover, another general problem is the low likelihood that health care systems would implement and disseminate unique prevention programs for each single disorder. Also, MDD and ADs are highly comorbid (Beekman et al., 2000), and risk factors and subclinical symptoms of depression and the different ADs overlap (de Graaf et al., 2002; Kessler et al., 1994). Transdiagnostic concepts aiming at preventing the incidence of both conditions simultaneously may therefore show much promise (Barlow et al., 2004; Brown and Barlow, 1992). Transdiagnostic interventions that target a set of shared risk factors might increase the potential reach and impact of psychological prevention programs (Titov et al., 2011).

Comorbidities can also be targeted through individual tailoring which allows participants to receive individualized treatment protocols based on symptom profiles, preference, need, or characteristics (Andersson and Titov, 2014; Păsărelu et al., 2017). This also means that participants following a treatment protocol do not all receive the same standardized treatment.

However, evidence for the potential of transdiagnostic individually tailored concepts that target the prevention of MDD and ADs in adults, so far, is scarce. Only very few RCTs have been conducted that evaluated the effects of a psychological intervention on the onset of depression and anxiety disorders, and results have been mixed (Austin et al., 2008; Joling et al., 2012; van't Veer-Tazelaar et al., 2009a). In one study (van't Veer-Tazelaar et al., 2009a) the effectiveness of an indicated stepped-care prevention program of depression and anxiety disorders was evaluated in an at-risk elderly population and found the incidence rate halved in a 12-month follow-up. However, others (Austin et al., 2008) found no significant differences between a cognitive behavioral group intervention and the wait control group (WCG, information booklet) when considering the prevention of depression and anxiety in an at-risk population of pregnant women. Another study (Joling et al., 2012) testing the effectiveness of family meetings for at-risk family caregivers of dementia patients, found no superiority in preventing depression and anxiety disorders. In sum, more research is needed to investigate the potential of transdiagnostic psychological interventions for the prevention of MDD and ADs.

The previously mentioned limitations of conventional prevention programs could be overcome by utilizing internet- and mobile-based interventions, e.g. IMIs. Main advantages include: (1) accessibility: they are easily accessible at any time from any location via the internet, (2) anonymity: participation is anonymous so that stigmatization or self-disclosure in group settings can be avoided, (3) flexibility: participation can be completed at any individual speed, and materials can be reviewed as often as desired, (4) personalization: the intervention is adaptable and can be tailored to specific needs, and (5) treatment availability: affected individuals can be reached earlier than through traditional mental health services. Also, (6) scalability: IMIs are easily scalable, as by applying a small increase of therapeutic resources a greater proportion of the eligible population can be reached.

IMIs have already been shown to be effective in reducing symptoms of diverse psychological disorders, such as the treatment of depression (Ebert et al., 2015a, Ebert et al., 2015b; Königbauer et al., 2017), anxiety (Cuijpers et al., 2009; Mayo-Wilson and Montgomery, 2013; Olthuis et al., 2015), sleep disorders (Cheng and Dizon, 2012), eating disorders (Dölemeyer et al., 2013), or alcohol use disorders (Riper et al., 2014).

However, evidence of the effectiveness in preventing the incidence of CMDs through internet interventions in public health settings is still scarce; the latest systematic review identified ten RCTs that evaluated the effect of an IMI on CMD onset (Ebert et al., 2017a, Ebert et al., 2017b). The only trial on the prevention of general anxiety did not result in significant positive findings (Christensen et al., 2014). Of the three primary prevention trials aiming to reduce the incidence of a depressive disorder (Buntrock et al., 2016a; Imamura et al., 2015; Thompson et al., 2015), only one trial (Buntrock et al., 2016a; Ebert et al., 2016a) used standard diagnostic procedures, while the other two trials relied only on self-report questionnaires for onset identification. The aforementioned study by Buntrock, Ebert and colleagues (Buntrock et al., 2014) showed that internet-based guided self-help for the indicated prevention of depression (including at-risk individuals) can reduce symptoms of depression in adults with subthreshold depression (Buntrock et al., 2015) and reduce the risk of MDD onset by 39% within a year (Buntrock et al., 2016a).

Even though it is often argued that internet-based interventions are likely to reduce the treatment costs, evidence from RCTs is still scarce and even completely absent when it comes to IMIs for the transdiagnostic prevention of both MDD and ADs. The only trial we are aware of evaluating cost-effectiveness of internet-based prevention is the trial from Buntrock and colleagues in which the authors showed that an IMI for indicated prevention has a high likelihood for cost-effectiveness, both from a societal and a public health care perspective (Buntrock et al., 2017). After the development of an intervention, the primary running costs are directly related to guidance time.

Previous research on guidance intensity and efficacy led to assume that internet-based mental health interventions without any kind of guidance were less effective than interventions including at least some guidance from a professional (Johansson and Andersson, 2012; Richards and Richardson, 2012; Spek et al., 2007). One study examining the role of support in internet-based problem-solving treatment for symptoms of anxiety and/or depression found that participants who received weekly support improved significantly in reducing depression compared to a WCG while participants with no guidance, guidance on demand, or guidance without training showed no significant improvement (Kleiboer et al., 2015). Results for the reduction of anxiety were less pronounced but still in favor of the weekly support condition. This notion is partly supported by a systematic review (Baumeister et al., 2014), identifying 14 articles fulfilling inclusion criteria, which showed superiority of guided internet-based interventions for mental health related problems in comparison to unguided treatment. However, the review and recent emerging evidence also propose that differences in outcomes between guided and unguided interventions are smaller than previously assumed, or might even be non-existent altogether (Dear et al., 2016; Königbauer et al., 2017; Mira et al., 2017; Olthuis et al., 2015).

A reason unguided IMIs might be preferable even when potentially less effective is that they could produce larger effects at a population level with regard to the reduction of disease burden, as many more individuals could be reached at comparable costs. The evaluation of (cost-)effectiveness, therefore, remains an open research question and more studies are needed directly comparing guided and unguided IMIs for mental health related issues (Boß et al., 2015; Ebert et al., 2014).

Study aims: This study aims at investigating the (cost-)effectiveness of a guided and unguided transdiagnostic individually tailored IMI for the prevention of MDD and AD in a clearly defined subclinical sample. The subclinical sample is defined by including individuals who show detectable symptoms of depression and anxiety indicated by a score of ≥16 on the Center for Epidemiological Studies Depression Scale (CES-D) (Hautzinger et al., 2012; Radloff, 1977) or score of ≥5 on the assessment of the Generalized Anxiety Disorder (GAD-7) (Spitzer et al., 2006), and who do not fulfill criteria for a full-blown AD or MDD as assessed in a diagnostic interview.

Specific aims include: 1) to evaluate the efficacy of guided and unguided IMIs to reduce the incidence of full syndrome CMDs compared to TAU, 2) to assess the efficacy of guided and unguided IMIs on symptom severity and other psychological outcomes when compared to TAU, 3) to compare the efficacy of guided and unguided IMIs on the reduction of symptom severity and other secondary continuous outcomes, 4) to assess the cost-effectiveness of guided and unguided IMIs, and lastly 5) to assess and compare the acceptability of guided and unguided IMIs.

2. Materials and methods

2.1. Objectives and hypotheses

We hypothesize 1) that both guided and unguided IMIs are more effective and cost-effective than TAU (no intervention); 2) guided IMIs to show higher effectiveness, lower cost-effectiveness but greater acceptability by the participants than unguided IMIs.

2.2. Participants

We will include individuals who…

  • -

    are at minimum 18 years of age, in the Dutch trial site individuals can participate from the age of 16,

  • -

    score ≥16 on the Center for Epidemiological Studies Depression Scale (CES-D) (Hautzinger et al., 2012; Radloff, 1977) or score ≥5 on the assessment of the Generalized Anxiety Disorder (GAD-7) (Spitzer et al., 2006),

  • -

    have Internet access and a valid email address,

  • -

    sufficient language skills in the considered program language in reading and writing,

  • -

    and are willing to give informed consent.

We will exclude individuals who…

  • -

    are currently on a waiting list for mental health treatment or have received psychological treatment in the previous 6 months for depression and/or anxiety,

  • -

    currently fulfill the criteria for a full syndrome MDD or AD according to the Mini International Neuropsychiatric Interview Version 5.0 (Mini) (Sheehan et al., 2006),

  • -

    fulfilled criteria of an MDD or AD in the previous 6 months and showed a main symptom (Dysphoria, Anhedonia) of MDD in the previous 3 weeks (to exclude individuals currently in remission of MDD),

  • -

    report to have been diagnosed with psychosis, bipolar disorder or experienced dissociative symptoms in the past,

  • -

    or show a high suicidal risk in the MINI during the diagnostic interview.

  • -

    Participate in similar studies at the time of inclusion (only in the Netherlands).

2.2.1. Recruitment

Participants will be recruited in Germany, Switzerland, Austria, Spain and the Netherlands. They are primarily recruited through cooperating health care insurances via their webpages as well as through members' magazines. Moreover, participants will also be recruited through an internationally organized mental health care risk assessment in university students (caring university project, WHO World Mental Health international college survey). Participants in the Netherlands are recruited through the latter, as well as additional recruitment strategies, such as flyers and posters on campus sites. The study website provides information about the ICare Prevent online training (icareprevent.com), the general study procedure, and details on how to sign up for participation. The trial is open to all individuals fulfilling the inclusion criteria and not restricted to the cooperating health care insurance companies and universities. Recruitment will begin in February 2017. Interested individuals can sign up for participation on the ICare Prevent website (icareprevent.com) or associated webpages, or contact the study administration team via email directly (training@icareprevent.com) by providing the research team with their email address and a name which can be a pseudonym if desired.

2.2.2. Assessment of eligibility and randomization

Individuals who apply for study participation will receive an email with detailed information about the study procedure. Applicants are asked to complete online screening questionnaires that assess the severity of their depression (CES-D ≥ 16) and their anxiety level (GAD-7 ≥ 5). The screening also includes questions on demographical information, assessment of history and current state of mental health, and past and current treatment history. Once the screening is completed and participants fulfill the inclusion and do not fulfill any exclusion criteria, participants receive detailed study information and a consent form. Participants in the Netherlands receive and fill-out the consent form before the screening. They are notified that they can withdraw from the intervention and/or study at any time without any negative consequences. They must sign the consent form and send it back via post or email. Once the signed consent is received, trained interviewers will conduct the diagnostic interviews, and individuals that fulfill the inclusion and none of the exclusion criteria will proceed to complete the baseline questionnaires. As soon as participants have completed all baseline questionnaires, they enter the study and are randomized to one of the three study conditions: guided intervention, unguided intervention, or TAU.

Participants are randomized to either one of the intervention groups (unguided intervention or guided intervention) or TAU in a 1:1:1 ratio. Randomization will be stratified according to trial site (Germany/Switzerland, Spain, and the Netherlands), and presence of subclinical symptoms (three strata: S1, subclinical anxiety; S2, subclinical depression; S3, subclinical anxiety and subclinical depression). A block-randomization algorithm with concealed block length will be used. The generation algorithm and resulting lists are validated by an independent biostatistician not otherwise directly involved in the study. The generation of randomization lists and the randomization of enrolled participants will be performed centrally for all countries by the ICare partner Westfälische-Wilhelms-Universität Münster (Institute of Biostatistics and Clinical Research, Westfälische-Wilhelms-Universität Münster, Germany) not directly involved in the recruitment or active study phase.

During the randomization process, allocation will be concealed from participants, researchers involved in recruitment and study administration, diagnostic assessors and eCoaches. Once randomization has been completed by the independent researcher, the study administration is informed about randomization outcome who inform the participants about the outcome (only whether they receive immediate or delayed access to treatment), and participants in the intervention groups then receive immediate access to the ICare Prevent training. The participants in the control group will receive the login data required to complete the training 12 months after randomization.

2.3. Study design

This trial will be conducted in compliance with the protocol, the Declaration of Helsinki, and good clinical practice. The trial is registered in the German Clinical Trial Registration (http://www.drks.de/drks_web/,DRKS00011099).

All procedures involved in the study will be consistent with the generally accepted standards of ethical practice approved by local ethics committees of each partnering country:

  • -

    Friedrich-Alexander University Erlangen-Nuremberg ethics committee (144_16 B),

  • -

    the Ethics Commission of the Canton of Bern (2016–01389),

  • -

    the Ethics Committee of the University of Valencia (H1476878127646),

  • -

    and the Medical Ethics Committee of the Vrije Universiteit Amsterdam (2017.131).

Data security/confidentially will be guaranteed; all relevant EU legislation and international texts on privacy will be observed and respected. Regarding regulation at international level, starting from the OECD guidelines including the “Guidelines on the protection of privacy and transborder flow of personal data” (1981) and “Guidelines for the security of information systems” (1991/92), the ICare consortium in particular acknowledges heterogeneity in international data protection jurisdiction.

A three-armed multi-country RCT will be conducted to compare guided and unguided IMIs with TAU. Trial sites are in Germany/Switzerland, the Netherlands, and Spain. Both active conditions are based on the same intervention ICare Prevent and differ only with regard to the guidance format (unguided and guided treatment). All intervention arms will have full access to TAU. TAU differs per country and region. In all countries, TAU is likely to be heterogeneous but this is a reflection of clinical practice. We will not interfere with nor manualize TAU. However, to control for potential confounding effects, TAU will be closely monitored by self-report assessments. By this, we mean that information on all types of medical help received during the duration of the study will be collected so that an accurate description of TAU will be available. After the initial screening, assessments will take place at baseline (T1), mid-intervention (after completion of module 5 or 5 weeks after randomization, T2), post-intervention (8 weeks after randomization, T3), at 6 months follow-up (T4), and at 12 months follow-up (T5). See Table 3 for a detailed overview of assessments. See Fig. 1 for a detailed overview of the study design.

Table 3.

Overview study self-report assessments.

Construct Questionnaire T0 T1 T2 T3 T4 T5
Demographics Socio-demographic data (SozDemo)
Psychopathology
Mental health
Mental health (PG)
Psychopathology
Depression
Center for Epidemiological Studies Depression Scale - Short Version (CES-D)
Psychopathology
Anxiety
General Anxiety Disorder Measurement (GAD-7)
Psychopathology
Depression
Patient Health Questionnaire (PHQ-9)
Psychopathology
Alcohol abuse
The Alcohol Use Disorders Identification Test (AUDIT-C)
Quality of life
Wellbeing
WHO-Five Wellbeing Index (WHO-5)
Quality of life EuroQol (EQ-5D)
Quality of life Assessment Quality of Life (AQoL-8D)
Quality of life
Health economic evaluation
Client Service Receipt Inventory (CSRI)
Training
Expectancy
Credibility/Expectancy Questionnaire (CEQ)
Training
Negative treatment effects
Inventory to assess negative effects of psychotherapy (INEP)
Training
Program evaluation
(Based on) Client Satisfaction Questionnaire (CSQ-8)
Training
Attitudes toward help
Attitudes Toward Seeking Professional Help (ATSPPHS)
Training
Supportive accountability
Supporting Accountability (SA)
Training
Treatment Satisfaction
(adaption of) Working Alliance Inventory SR (WAI-SR)
Risk factors
Risk factors questionnaire
Risk Factor Questionnaire (developed by ICare Prevent team) (RF)
Risk factor
Worry
Penn State Worry Questionnaire – Ultra Brief (PSWQ-3)
Risk factor
Sleep
Pittsburgh Sleep Quality Index (PSQI)
Risk factor
Self-esteem
Rosenberg Self Esteem Scale (RSES)
Risk factor
Resilience
Connor-Davidson Resilience Scale (CD-RISC)
Other
Emotion regulation
Subscales of emotion regulation scale questionnaire (SEK-15)
Other
Incongruence
Incongruence questionnaire (short version) (INK-K)
Other
Personality
Big Five Inventory (BFI-10)
Other
Self-regulation
Self-Regulation Scale (SSRQ)
Other
Behavioral activation
The Behavioral Activation for Depression Scale (BADS-FS-9)
Other Drop-Out Reasons (developed by ICare Prevent team) (DG)
Other Other help (developed by ICare Prevent team) (INH)

T0 = screening, T1 = baseline, T2 = 5 weeks after randomization, T3 = post-intervention, 8 weeks after randomization, T4 = 6-month follow-up, T5 = 12-month follow-up.

Trial site Netherlands will assess the CES-D only at screening and will not assess ATSPPHS, SSRQ and SEK, participation will be optional in the risk factor questionnaire.

Fig. 1.

Fig. 1

Study Design.

2.4. Intervention

The transdiagnostic individual tailored ICare Prevent online training aims at reducing depressive complaints and anxieties, reduce associated risk factors, and strengthen protective factors. The training is internet-based and supported by a mobile app. Transdiagnostic elements of the intervention include psychoeducation on etiological and maintaining factors of depressive and anxiety symptoms, and targeting common risk factors. Furthermore, the intervention can be individually tailored to the specific need of the individual as participants can choose to focus on problem solving, recommended for persons with depressive mood, or practice exposition to fear-inducing situations, aimed at individuals suffering from anxieties, and participants can continuously choose to focus on topics of interest throughout the intervention. The transdiagnostic individual tailored approach enables to target both depression and anxieties, but also to tailor the treatment to other individual preferences and risk factors. One main strategy of the intervention, commonly found in internet interventions, is the concept of self-help. To foster self-help techniques, participants are encouraged to integrate and practice newly acquired strategies in daily life through transfer tasks (homework assignments). The intervention builds on adaptions of evidence-based modules from an a range of previous interventions which have shown to be effective in the prevention of MDD (Buntrock et al., 2016b; Ebert et al., 2016), treatment of depression (Buntrock et al., 2015; Ebert et al., 2017a, Ebert et al., 2017b; Ebert et al., 2016b; Nobis et al., 2015), and sleeping problems (Ebert et al., 2015a, Ebert et al., 2015b; Thiart et al., 2015, Thiart et al., 2016).

The intervention consists of eight modules: addressing needs (session 1), behavioral activation (session 2), psychoeducation (session 3), cognitive restructuring (session 4), problem solving or exposure (session 5 and 6), planning for the future (session 7) and a booster session (session 8) (see Table 1 for an overview of the main modules). Participants can choose between two disorder-specific modules in session 5: problem solving (training for participants who want to focus primarily on the reduction of depressive symptoms) and exposure (for participants who want to focus on the reduction of symptoms of anxiety). Another major element is that participants can choose between several elective modules (integrated into sessions 2 to 7) based on individual need and/or preference. The elective modules are directed at rumination & worries, acceptance, relaxation, alcohol & affect regulation, self-worth, perfectionism, appreciation and gratefulness, and sleep (see Table 2 for an overview of the elective modules). Each session can be completed in approximately 45 to 60 min. We advise participants to do at least one and a maximum of two sessions per week. Further content can only be accessed once previous sessions are completed. Consequently, the training lasts about 4 to 7 weeks plus one additional booster session 4 weeks after completion of the last session. The booster session aims to help participants transfer acquired strategies into daily life. Sessions consist of texts, testimonials, audios, short educational video clips, and also include interactive elements such as quizzes, exercises, and homework. Participants are also encouraged to keep an online diary at certain intervals of the training which can be accessed either via the web interface or an optional smartphone app. The intervention uses content tailoring, thereby engaging the participant by encouraging them to make real-time choices among various response options, and then providing individualized content based on specific needs or preference. The training is built on responsive web-design and can be completed on any kind of internet-ready device, such as PCs/laptops, smartphones, or tablets. Participants can follow narrated lessons by activating the read-aloud function. Participants can also opt to receive motivational messages and small exercises referred to as Smart Coach. These messages are sent directly to their mobile device. They can choose between a light version (3 messages per day) and an intensive version (5 messages per day). These notifications will support the participants in transferring the exercises of the training into their daily lives (e.g., short relaxation exercises: “Relax your muscles in your hands and arms for 3 seconds now. Follow your breathing and each time you breathe out, relax a little more.”).

Table 1.

Content of the ICare Prevent training.

Intervention content Session
Addressing needs 1
Behavioral activation 2
Psychoeducation 3
Cognitive restructuring 4
Problem Solving or Exposure 5
Problem Solving or Exposure 6
Plan for the future 7
Booster session 8

Table 2.

Content of the ICare Prevent elective modules.

Rumination & worries: Dealing with excessive worrying & reducing rumination habits
Acceptance: Acceptance of undesired affective states
Relaxation: Progressive muscle relaxation
Alcohol & affect regulation: Reducing affect regulation related alcohol consumption
Self-worth: Acknowledging and strengthening self-worth
Perfectionism: Dealing with perfectionism
Appreciation & gratefulness: Practicing mindfulness strategies
Sleep: Improving sleep hygiene and practicing sleep restriction to enhance sleep quality

2.4.1. Addressing needs (session 1)

In the first session, participants receive an introduction to the training and familiarize with the session content as well as the core elements, such as testimonials, diaries, and videos. Motivation is strengthened by assessing personal treatment motivation, and participants are asked to formulate realistic, individual and concrete goals which can be modified and revised throughout the training. One main focus of the intervention lies on reducing personal incongruence in daily life, by supporting participants to achieve a balance between personal needs and values, and behavior based on the congruence theory by Grawe, 2004. Hence, participants are educated on the relationship between personal needs and values, and psychological well-being, and then work on identifying their unfulfilled needs and core values. They are encouraged to plan activities which serve the purpose of strengthening life areas important to them, and progress and difficulties can be documented in the activity diary.

2.4.2. Behavioral activation (session 2)

In the second session, participants are presented with strategies on how to deal with difficulties concerning the implementation of planned activities and how to reduce avoidant behaviors. In addition to behavioral activation strategies related to core personal needs and values addressed in session one, they are also introduced to the concept of mood stabilizing reinforcing activities that can be used to overcome lowered mood, and participants are encouraged to plan exercising self-chosen pleasant activities in their daily life.

2.4.3. Psychoeducation (session 3)

The participants are provided with psychoeducational information on depression and anxiety, and the multifactorial causes explaining etiology and maintenance. Participants are encouraged to identify symptoms they have previously experienced or are currently dealing with. Information on the different forms of depression and anxiety including diagnosis criteria is provided. To enhance the understanding of the disorders, influencing factors benefitting the development and maintenance of depression and anxiety, risk factors, and the vicious circles of upkeep are explained. After receiving general information, participants can choose throughout the session whether to review detailed information on one of the disorders (depression or anxiety) or work through both. Participants are asked to identify their symptom development and describe the course of their symptomology.

2.4.4. Cognitive restructuring (session 4)

In the fourth session, participants receive information on the causal relationship between cognitions and emotions. Participants are introduced to a thought record in which situations are divided into three components: situations, thoughts, and consequences. Participants are given the opportunity to individualize their personal thought records by including personal situations, cognitions, and emotions. The importance of negative emotions is also stated. In the second phase of the thought record, participants are motivated to question the reality basis of their negative thoughts, encouraged to find positive and helpful thoughts, and to generally concentrate on and practice positive thinking habits.

2.4.5. Problem solving or exposure (sessions 5 & 6)

Participants can choose whether they wish to work on 1) acquiring and enhancing problem solving skills to reduce depressive symptomology, or 2) practicing exposition to fear-inducing stimuli and situations to reduce feelings of anxiety.

Participants learn the distinction between solvable and unsolvable problems and are asked to assign their own problems to these two categories. They are introduced to the concept of a 6-step problem-solving plan in which they: 1) identify a problem, 2) define a realistic target state, 3) find possible solutions and choose the seemingly best solution, 4) create a detailed solution plan, 5) practice a plan, and 6) verify the implementation success. Participants are expected to practice their plan during the following week in real life situations, and are encouraged to analyze the success or failure of it in the following session. They can also opt to review and adapt their plan in the following session.

Participants who wish to reduce anxieties are introduced to the concept of avoidant and safety behavior, how confrontation of fear can lead to habituation, and practice exposure to fear-inducing situations. Participants are educated on the importance of confronting fears through exposition, what actually happens during exposition, and why it is more effective in the long term perspective for enhancing well-being than avoidance strategies. Participants are asked to identify perceived difficult and fear-inducing situations, and rate their expected feelings of anxiety in relation to these situations. They are introduced to strategies to deal with and work on overcoming their anxiety by exposure to fear-provoking situations. It is recommended to start with less threatening situations and once these seem manageable to work on more severe situations. Participants are introduced to a challenge management protocol, which intends to prepare them for the situations they aim to work through. In the following session, they can evaluate their implementation success, identify personal safety behavior, and focus on facing more challenging situations. Participants are encouraged to create a plan for the upcoming weeks to practice exposition to more fear-inducing situations.

2.4.6. Plan for the future (session 7)

In the last training session, participants can review brief summaries of each session. They are asked to reflect on progress concerning their training goals, and to identify mechanisms of the training that helped them. They are also given the opportunity to write a letter to their future selves assuming they have completed four more weeks of practice of acquired strategies. Subsequently they are encouraged to make a plan of specific strategies they want to continue to train in everyday life until the booster session, in order to maintain and generalize acquired strategies.

2.4.7. Booster session (session 8)

The participants are invited to complete a booster session four weeks after completion of the seventh session. In this session, they can reflect on their learning experience and personal goal attainment. They are asked to reflect on their current state of mental health, and provided with additional information on support, if needed. Participants can review the letter they wrote to themselves in the last training session, can reformulate their goals and are asked to make plans to implement their intentions in the coming months.

2.4.8. Support

2.4.8.1. Guided treatment

During the active intervention phase participants of the guided treatment arm are supported by an eCoach who provides individual manualized feedback after completion of each session. eCoaches will have at least a Bachelor's degree in Psychology and have access to supervision when required. Guidance is manualized, eCoaches use preformulated standardized text blocks that are prepared for every lesson. The standardized text blocks are individually adapted to the clients input and overall progress. The eCoaches communicate with participants through the internal messaging function of the platform. Estimated mean time per feedback is 20 min. eCoaches are advised not use >30 min per individual feedback. Approximately 2.5 h is the total time an eCoach will spend per participant. Session adherence is also monitored, when participants fail to complete a session within 7 days, eCoaches will send out email reminders (after 7, 14, and 21 days). Additionally, after 28 days a reminder will be sent via text message (not in the Netherlands NL). Both personal and automatic reminders have shown to improve adherence to self-guided health promotion and behavior change interventions (Fry and Neff, 2009; Titov et al., 2013).

2.4.8.2. Unguided treatment

Participants of the unguided treatment arm do not receive any individualized feedback, however, after completion of a session, they will receive encouraging standardized messages automatically sent through the training platform. Session adherence is monitored and participants are also provided with messages based on non-adherence. Should participants fail to complete a session within 7 days, they will receive standardized email reminders (after 7, 14, 21 days) by the study administration team encouraging them to continue working on the current module. If participants fail to respond within 28 days, they will receive a reminder via text message (not in NL).

All study participants can contact a support email address (support@icareprevent.com) in case of any technical difficulties with the training or the assessments.

2.5. Assessment and data management

Data management and monitoring will be provided by the Institute of Biostatistics and Clinical Research (Westfälische Wilhelms-Universität Münster) for the whole ICare consortium in order to maintain comparable high quality in the conduct of the ICare research projects in trial planning, data management, online monitoring, and analysis.

Self-reports and observer-based assessments will take place at screening (T0), baseline (T1), mid-intervention (T2, after completion of module 5 or 5 weeks after randomization), post-intervention (T3, 8 weeks after randomization), and at 6- (T4) and 12-month follow-ups (T5). Observer-based assessments from clinical diagnostic interviews will be carried out via telephone. Self-report data will be collected using a secured online-based assessment system (AES, 256-bit encrypted).

2.5.1. Observer-based assessments

Diagnostic clinical interviews will be conducted at baseline, at post-intervention, and at 6- and 12-month follow-up. The interviews consist of an adaption of the MINI focusing on depression and any anxiety disorder (Mini 5.0) (Sheehan et al., 2006), the Quick Item Inventory of Depressive Symptomatology clinician rating (QIDS-C) (Rush et al., 2003), and structured interview of the Hamilton Anxiety Rating Scale (SIGH-A) (Hamilton, 1959; Shear et al., 2001). The MINI will not be assessed at post-intervention.

All clinical interviews will be conducted by trained master psychology students or experienced psychotherapists who are blind to treatment condition. The interviews will be recorded and approximately 10% of the cases will be assessed by another rater to examine interrater reliability. Disagreement shall be solved by discussion and the agreed rating will be used for analysis. If this is not possible, the assessment will be rated by an experienced psychotherapist (gold standard) and this rating will be used for analysis. Interrater reliability will be reported for each trial site (Germany/Switzerland, Spain and the Netherlands) individually. Measures to assure blinding include: (a) explanation to the participant the importance of keeping the assessors blinded to randomization status; (b) verbal reminder to the participant before each assessment; and (c) documentation after each assessment, whether the interviewer is still blind to treatment condition. In case of evidence for blinding break down, the interviewer will be changed to the next outcome interview. The interviewers will be asked to guess each participant's randomization status. These guesses will be compared with the actual status. Cohen's kappa will be computed to clarify whether hit rates differ from what can be expected from chance.

2.6. Outcomes

2.6.1. Primary outcome

Primary outcome will be time to CMD onset (any anxiety/depression disorder) in guided and unguided IMI compared to TAU within a 12 months follow-up period assessed by the MINI 5.0 (Sheehan et al., 2006). The Mini will be assessed at baseline, 6 months follow-up (FU) and 12 months FU. The frequency of outcome interviews allows for a temporal precision of onset of CMD within the inspected timeframe. The exact time to CMD onset (given in weeks since randomization) will be assessed as accurately as possible using the Life Chart method developed by Lyketsos et al. (1994) in order to reduce a potential recall bias. In this method, age- and calendar-linked personal landmarks are used to assess the time sequence of i.e. depressive symptomatology and life events in parallel. During the interview the first day of an episode will be established. If the exact day could not be established, the closest week (month) will be defined and the mid-point of that week (month) will be used.

2.6.2. Secondary outcomes

2.6.2.1. Observer-based

Observer-rated depressive symptom severity is assessed by clinician ratings by the QIDS-C (Rush et al., 2003) at baseline, 8 weeks post-intervention, at 6- and 12-month follow-up. It encompasses the following nine criteria in 16 items: sleep, sad mood, appetite/weight, concentration/decision making, self-view, thoughts of death or suicide, general interest, energy level, and restlessness/agitation. Each item is answered on a scale from 0 to 3, and total scores range from 0 to 27 (Rush et al., 2003). A meta-analysis showed Cronbach's alpha (α) was found to be acceptable (α = 0.69–0.89) (Reilly et al., 2015).

Observer rated anxiety symptom severity will be assessed using the SIGH-A (Shear et al., 2001) at baseline, 8 weeks post-intervention, at 6- and 12-month follow-up to assess the severity of anxiety. It consists of 14 items rated from 0 to 4, 4 being the most severe. The score is summed to receive an overall rating of anxiety severity ranging from 0 to 56. Test-retest reliability of the SIGH-A was 0.89 (Shear et al., 2001).

2.6.2.2. Self-report

The complete list of self-report measures and measurement time points are shown in Table 3.

2.6.2.3. Psychopathology

Self-reported depressive symptoms will be measured with the Center for Epidemiological Studies Depression Scale (CES-D) (Radloff, 1977). This frequently used self-report instrument consists of 20 items that are answered on a four-point Likert scale referring to the previous week. Total scores range from 0 to 60. The internal consistency of this measure has been found to be excellent (α = 0.95) (Lehr et al., 2008). Depressive symptoms are also measured by the Patient Health Questionnaire (PHQ-9) (Kroenke et al., 2001) and consist of 9 items. Each item covers one of the nine DSM-IV criteria for the diagnosis of a major depression. This instrument has a high internal reliability (α = 0.89) and also test-retest reliability with 0.84 (Löwe et al., 2004; Martin et al., 2006; Spitzer et al., 2006), and shown to have a good interformat reliability to paper, when delivered online (Erbe et al., 2016).

The generalized anxiety disorder measurement (GAD-7) (Spitzer et al., 2006) is used to assess GAD and symptom severity. It consists of 7 items answered from 0 (not at all) to 3 (nearly every day). It possesses excellent internal consistency (α = 0.92) and good test-retest reliability (intraclass correlation (ICC) = 0.83) (Löwe et al., 2008; Spitzer et al., 2006).

Substance abuse will be assessed by a 3 item alcohol screen (AUDIT-C) (Bush et al., 1998). Each item is answered on a 5-point Likert scale and the total score ranges from 0 to 12. This measurement has acceptable psychometric properties (α = 0.61, ICC = 0.89) (Bradley et al., 2007; Dybek, 2008).

Worry will be assessed by the 3 item Penn State worry questionnaire ultra brief (PSWQ-3; range 0–18, α = 0.85) (Berle et al., 2011; Meyer et al., 1990)

Sleep quality will be assessed by the 1 item subscale of the Pittsburgh sleep quality index (PSQI) (Buysse et al., 1989), answered on a 4-point Likert scale. Psychometric properties are acceptable (α = 0.74, ICC = 0.86) (Mondal et al., 2013; Rener-Sitar et al., 2014)

2.6.2.4. Quality of life

To measure quality of life we will use the Assessment of Quality of Life Instrument (AQoL-8D) (Richardson et al., 2014; Richardson and Rothstein, 2008), which consists of 35 items. They entail eight dimensions: independent living (α = 0.9, ICC = 0.86), pain (α = 0.85, ICC = 0.86), senses (α = 0.69, ICC = 0.51), mental health (α = 0.84, ICC = 0.89), happiness (α = 0.85, ICC = 0.90), coping (α = 0.80, ICC = 0.79), relationships (α = 0.73, ICC = 0.88), self-worth (α = 0.85, ICC = 0.81) (Richardson et al., 2014). Psychometrics properties are, therefore, acceptable.

Additionally, the EuroQol (Group, 1990), consisting of five dimensions (EQ-5D-5 L) and a visual analogue scale for self-rated health state, will be applied to measure health-related quality of life. The dimensions consist of mobility, self-care, usual activities, pain/discomfort and are answered on a 5-point scale from 1 (no problems) to 5 (severe problems). Psychometric properties are acceptable (α = 0.7, ICC = 0.64) (King et al., 2009; Luo et al., 2003).

The AQoL-8D and EQ-5D-5 L ratings will be used in the economic evaluation.

2.6.2.5. Well-being

Well-being will be assessed by the 5-item WHO-5 Well-Being Index (WHO-5) (World Health Organisation, 1998) answered on a 0 through 5 scale, and scores range from 0 to 30 (α = 0.82) (de Wit et al., 2007).

2.6.3. Health economic evaluation

Key to the cost-effectiveness and cost-utility analyses is information on the impact of the intervention on publicly funded services and the wider society. Individuals' use of services and supports will be recorded on an adapted client service receipt inventory (CSRI) (Beecham and Knapp, 2001) alongside wider societal impacts such as productivity loss. Unit costs will be estimated for each service and support used following well-established theory-driven methods and multiplied by the use made be each individual to obtain a cost per person (Beecham, 2000). Costs will be expressed in the same currency across countries using OECD purchasing power parities data

2.6.4. Training and acceptability

User satisfaction will be assessed by a questionnaire based on the Client Satisfaction Questionnaire (CSQ-8) (Attkisson and Zwick, 1982), adapted to assess user satisfaction in online interventions (Boß et al., 2016). Global client satisfaction with internet-based training is measured by 8 items. Previous research indicated good psychometric properties (α = 0.84–0.97) (Matsubara et al., 2013).

Other measures related to the training include participant expectation (CEQ) (Borkovec and Nau, 1972; Devilly and Borkovec, 2000), satisfaction with treatment, an adaption of the Working Alliance Inventory (WAI) (Horvath and Greenberg, 1989), negative effects of the treatment (INEP) (Ladwig et al., 2014), and attitudes toward seeking professional psychological help (ATSPPH) (Fischer and Farina, 1995; Surgenor, 1985). Lastly, the occurrence of any negative effects will be identified as in Rozental et al. at the 12 months follow-up (Rozental et al., 2015).

2.6.5. Response

To determine the numbers of participants achieving a reliable positive outcome, we will code participants as responders or non-responders according to the widely used reliable change index (RCI) (Jacobson and Truax, 1991) after participation in the intervention. Participants will be considered responders when they display an RCI score of above 1.96.

2.6.6. Other assessments

Other assessments include demographics (e.g., age, gender, occupation, level of education etc.), current and previous mental health issues, current and previous experience with psychotherapy, personality dimensions measured by the ultra-short version of the Big Five Inventory (BFI-10) (Rammstedt and John, 2007), behavioral activation (BADS-FS) (Fuhr et al., 2016; Kanter et al., 2012; Martell et al., 2001), self-regulation (SSRQ) (Carey et al., 2004), self-esteem (RSES) (Rosenberg, 1965), resilience (CD-RISC) (Connor and Davidson, 2003), motivation (TEQ) (Ryan et al., 1995), various emotion regulation skills (subscales comprehension, acceptance, and emotional self-support by the German Emotion Regulation Scale Questionnaire, ERSQ-27, ERSQ-ES-GD) (Berking and Znoj, 2008; Ebert et al., 2013).

Additionally, risk factors for the onset of depression and anxiety, e.g. aspects of physical health, traumatic childhood and life events, family psychopathology, social environment, are assessed at baseline in order to investigate moderators of treatment outcome.

Adherence will be tracked by the system based on session completion.

2.6.7. Qualitative interviews

At least 10 participants with predominantly subclinical depression and anxiety symptoms that have been randomized to the intervention condition who have completed the intervention or who dropped out, will be asked to participate in qualitative interviews. The qualitative interviews aim at collecting subjective information on factors regarding training efficacy and adherence. These type of interviews can help improve IMIs with the goal of heightening engagement. Aspects assessed by the interview will include a) participants hope of improvement and expectations, b) perceived usefulness of the training, c) reasons for treatment success and failure, d) reasons for training adherence and non-adherence, e) usability and layout, f) quality and content of the intervention and received guidance, and g) overall strengths and limitations of the training. The analysis of the interviews is based on the qualitative content analysis by Mayring and Kuckarts (Kuckartz et al., 2009; Mayring, 2010).

2.7. Statistical methods

Data will be analyzed on an intention-to-treat basis including all participants who will be randomly assigned to conditions. Additionally, a per protocol analysis (PPA) will be conducted including only participants satisfying treatment protocol and assessments.

Time to onset of CMD between unguided and guided ICare Prevent and TAU will be analyzed by a two-sided log rank tests. We will test the two primary statistical hypotheses, i.e. (i) guided training vs. TAU and (ii) unguided training vs. TAU. The emerging multiple testing problem will be considered appropriately, by using a Bonferroni adjustment of local significance levels. Comparing survival curves instead of just onset rates allows a more precise estimation of the intervention effects and also increases power, thereby allowing for the detection of smaller effects. Missing data will be handled by censoring.

For differences in the change of depression and anxiety symptom severity multi-level mixed models regression analyses will be used. Depending on the distribution of data in the various outcome parameters we will choose either linear models (for normally distributed data) or negative binomial models (for left-skewed data). Each model includes assessment time as a predictor. The influence of potential predictive factors will be investigated by adding corresponding interaction terms to the model. All analysis will first be adjusted for country, type of subclinical symptoms, baseline symptom severity, history of mental health disorder, previous treatment experience and potential comorbid disorders. Missing data will be handled using multiple imputation.

We will also conduct a number of pre-planned exploratory subgroup analyses such as a) per country, b) per disorder, c) with/without comorbidity (depression & any anxiety; depression & alcohol abuse; any anxiety & alcohol abuse; combination of different specific anxiety disorder symptoms anxiety comorbidities). d) Moreover, we will create a multivariate risk prediction algorithm based on baseline risk factors and in which we will test the assumption that individuals with high vs. low predicted risk show a differential treatment effect.

For all analyses on continuous measures, Cohen's d (Cohen, 1977) will be calculated by standardizing the differences between baseline and follow-up scores by the pooled standard deviation of the baselines scores. The response rates will be compared across conditions with the help of contingency tables and Chi-Squared tests. The global significance level of the two null hypotheses tested within the primary analysis of the trial will be maintained by Bonferroni correction. Thus, the local significance levels are 2.5%. For the explorative analysis of the data, the significance levels will be set to 5%. Thus, p ≤ 0.05 will indicate the rejection of the respective null-hypothesis. The number needed-to-treat (NNT) with ICare Prevent (guided and unguided) to prevent one case of a CMD, respectively to achieve one response as compared to the control group, will be calculated.

The health-economic evaluation will involve a combination of a cost-effectiveness analysis (CEA) and a cost-utility analysis (CUA). The economic evaluation will be performed from a societal perspective (all relevant costs) and public sector costs with a time horizon of 12 months. In the CEA, the incremental cost-effectiveness ratio (ICER) will be expressed as the incremental costs per additional disorder free participant. In the CUA, the ICER will be expressed as incremental costs per quality-adjusted life year gained (QALY) as based on the EQ-5D-5 L. Bootstrapping (5000 times) will be used to quantify the uncertainty around the ICER. The bootstrapped ICERs will be plotted on a cost-effectiveness plane where the horizontal axis reflects differences in effects and the vertical axis differences in costs. The bootstrapped ICERs will also be shown in a cost-effective acceptability curve disclosing the probability that the intervention is cost-effective for a range of willingness-to-pay ceilings. To test the robustness of the base-case findings, a multi-way sensitivity analysis will be conducted. Several assumptions made in the base-case scenario (eg. QALYs based EQ-5D instead of the AQoL-8D) will be changed to assess their impact on the ICER.

The statistical analysis of the primary and secondary outcomes will be described in a statistical analysis plan that will be signed by the study committee and the responsible statistician.

2.8. Sample size calculation

In primary analysis we aim to explore the effects of the intervention on time to disorder onset. Based on meta-analytic evidence on the prevention of depression (van Zoonen et al., 2014) and randomized trials on combined prevention of depression and anxiety (van't Veer-Tazelaar et al., 2009b), we expect a mean 12-month survival rate (i.e., no disorder onset) in the control group of 75% (25% incidence rate), and 85% (15% incidence rate) in the active conditions, which translate in an absolute risk reduction of 10% and a relative risk reduction of 40%. This is a more conservative estimation than 50% relative risk reduction found in the only trial that has to the best of our knowledge been conducted on the combined indicated prevention of depression and anxiety. To detect such a difference between the active conditions and the control condition with a power of 80% and a significance level of 0.05, 254 participants are needed per group (762 in total). Accounting for 20% drop-out, 954 participants in total need to be recruited (n = 363 per group).

Based on the mean results from studies evaluating interventions aimed at the indicated prevention of a CMD (Allart-Van Dam et al., 2003; Clarke et al., 1995; de Jonge et al., 2009; Sheffield et al., 2006; van't Veer-Tazelaar et al., 2009b; Willemse et al., 2004) we expect an effect of d = 0.35 (Cuijpers et al., 2014a); difference in the reduction in secondary outcome depression severity between the active groups and the control group. So far evidence for internet-based interventions for subclinical symptoms of anxiety is lacking. Meta analytic evidence for internet interventions targeting anxiety disorders indicates an effect of d = 0.90 for internet-based guided self-help treatments and d = 0.58 for interventions without guidance. Given that the present trial targets individuals with subclinical symptoms with less room for improvement, much lower effects can be expected. Hence, we expect an effect size of d = 0.30.

3. Discussion

Depression and anxiety disorders are greatly prevalent globally and highly comorbid, associated with a decline of quality of life, and linked to substantial economic costs. IMIs could be an attractive, efficient and cost-effective approach to prevent the development of CMDs, by reducing the onset probability of full syndrome CMDs, and targeted at at-risk population transdiagnostic IMIs could help reduce overall disease burden.

To date, few RCTs have been conducted on the prevention of CMDs, and data for transdiagnostic treatment protocols are scarce. A remaining open research question is whether guided or unguided interventions are more cost-effective when implemented on a large scale. After the development of internet-based interventions, the most prominent cost factor is substantially linked to guidance time, and this relationship has not yet been examined in the context of interventions aimed to prevent the incidence of CMDs.

Overall, the present study will be the first to provide evidence for the efficacy and cost-effectiveness of transdiagnostic preventive interventions and provide valuable information on the optimal trade-off between treatment outcome and economical costs.

The present study has several strengths, including the strong methodology of a randomized controlled design with the direct comparison of two active conditions. The assessments include diagnostic outcome interviews as an objective addition to the self-report measurements. Also, there is an appropriate statistical analyses plan and missing data will be handled with state of the art methods. The treatment protocol is transdiagnostic and individual tailored adding to empirical evidence of treatment options targeting a heterogeneous at-risk group. Lastly, the online training is technically advanced as it relies upon content tailoring which allows the participant to make real-time choices which trigger different content based on preference or need.

This study also has some limitations. First, as the study is powered to detect differences between the active groups compared to the TAU group in the primary outcome onset of CMDs, the sample size will be insufficient to draw reliable conclusions on the differential effectiveness of guided vs. unguided IMIs regarding disorder onset. It will remain unclear which participants are likely not to profit from unguided treatment but would potentially profit from guided IMIs. Second, although pooling data from the four participating countries increases the feasibility of reaching the necessary sample size, differences between participants of the different trial sites need to be considered when interpreting the results which could contain larger variance than if conducted solely in one trial site. However, as the study is embedded in the ICare consortium whose goal it is to implement, to disseminate, and to increase acceptability of internet-based mental health interventions across Europe, it is a necessary risk and will also provide valuable information on the overall effectiveness of IMIs for the prevention of CMD in heterogeneous settings in Europe. Third, assessments take place only at post-intervention and follow-up time points; there will be no ambulatory assessment. Therefore, the results will have to be interpreted in relation to these predefined time points. As the intervention is time-consuming in itself and the assessments can be tedious, it was decided not to burden participants any further and possibly risk drop-out with weekly or even daily assessments, and the diagnostic interviews allow for temporal precision determining onset. And lastly, the study has an elaborate screening process. This poses the risk that highly motivated individuals will be more likely to participate in the intervention. In future, interventions must be implemented with low threshold access, in order to estimate the effects when delivered under routine care conditions.

4. Conclusion

To overcome the gap between the need for economical effective strategies for preventing mental health disorders and evidence-based treatment availability, (cost-)effective, low-threshold interventions, accessible to as many people as possible, are needed. Internet- and mobile-based mental health interventions are a promising strategy to overcome some of the limitations of face-to-face mental health interventions. This study will enhance the evidence-base for IMIs and provide information on the differential efficacy, and cost-effectiveness of two guidance forms of ICare Prevent.

Trial status

The first participants were enrolled in the study on February 1, 2017 Follow-up assessments for the remaining patients are expected to be completed by September 1, 2019.

Competing interests

David Ebert and Matthias Berking are stakeholders of the ‘Institute for Online Health Trainings’ that aims to transfer scientific knowledge related to the present research into routine healthcare.

Funding

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 634757.

Author contributions

DDE designed the study in the first place. DDE, DG, TB, TK, CTM, ACZ, KKW contributed significantly to the study design. DDE and KKW wrote the first draft of the manuscript. All authors contributed feedback, read, and approved the final manuscript.

Acknowledgements

We would like to thank Marvin Franke and Eva Dull for their support in developing the intervention, and all our participants without whom this study would not be possible.

References

  1. Allart-Van Dam E., Hosman C.M.H., Hoogduin C.A.L. The coping with depression course: short-term outcomes and mediating effects of a randomized controlled trial in the treatment of subclinical depression. Behav. Ther. 2003;34:381–396. [Google Scholar]
  2. Andersson G., Titov N. Advantages and limitations of internet-based interventions for common mental disorders. World Psychiatry. 2014;13:4–11. doi: 10.1002/wps.20083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Andrews G., Issakidis C., Sanderson K.Y., Corry J., Lapsley H. Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders. Br. J. Psychiatry. 2004;184:526–533. doi: 10.1192/bjp.184.6.526. [DOI] [PubMed] [Google Scholar]
  4. Attkisson C.C., Zwick R. The client satisfaction questionnaire. Eval. Program Plann. 1982;5:233–237. doi: 10.1016/0149-7189(82)90074-x. [DOI] [PubMed] [Google Scholar]
  5. Austin M.P., Frilingos M., Lumley J., Hadzi-Pavlovic D., Roncolato W., Acland S., Saint K., Segal N., Parker G. Brief antenatal cognitive behaviour therapy group intervention for the prevention of postnatal depression and anxiety: a randomised controlled trial. J. Affect. Disord. 2008;105:35–44. doi: 10.1016/j.jad.2007.04.001. [DOI] [PubMed] [Google Scholar]
  6. Barlow D.H., Allen L.B., Choate M.L. Toward a unified treatment for emotional disorders. Behav. Ther. 2004;35:205–230. doi: 10.1016/j.beth.2016.11.005. [DOI] [PubMed] [Google Scholar]
  7. Baumeister H., Reichler L., Munzinger M., Lin J. The impact of guidance on internet-based mental health interventions - a systematic review. Internet Interv. 2014;1:205–215. [Google Scholar]
  8. Baxter A.J., Vos T., Scott K.M., Ferrari A.J., Whiteford H.A. The global burden of anxiety disorders in 2010. Psychol. Med. 2014;44:2363–2374. doi: 10.1017/S0033291713003243. [DOI] [PubMed] [Google Scholar]
  9. Beecham J. 2000. Unit Costs: Not Exactly Child's Play. [Google Scholar]
  10. Beecham J., Knapp M. Costing psychiatric interventions. Meas. Mential Heal. Needs. 2001:200–221. [Google Scholar]
  11. Beekman A.T., de Beurs E., van Balkom A.J., Deeg D.J., van Dyck R., van Tilburg W. Anxiety and depression in later life: co-occurrence and communality of risk factors. Am. J. Psychiatry. 2000;157:89–95. doi: 10.1176/ajp.157.1.89. [DOI] [PubMed] [Google Scholar]
  12. Berking M., Znoj H. Entwicklung und Validierung eines Fragebogens zur standardisierten selbsteinschätzung emotionaler Kompetenzen (SEK-27) Z. Psychiatr. Psychol. Psychother. 2008;56:141–153. [Google Scholar]
  13. Berle D., Starcevic V., Moses K., Hannan A., Milicevic D., Sammut P. Preliminary validation of an ultra-brief version of the Penn State worry questionnaire. Clin. Psychol. Psychother. 2011;18:339–346. doi: 10.1002/cpp.724. [DOI] [PubMed] [Google Scholar]
  14. Berto P., Ilario D.D., Ruffo P., Di Virgilio R., Rizzo F. Depression: cost-of-illness studies in the international literature, a review. J. Ment. Health Policy Econ. 2000;3:3–10. doi: 10.1002/1099-176x(200003)3:1<3::aid-mhp68>3.0.co;2-h. [DOI] [PubMed] [Google Scholar]
  15. Birnbaum H.G., Kessler R.C., Kelley D., Ben-Hamadi R., Joish V.N., Greenberg P.E. Employer burden of mild, moderate, and severe major depressive disorder: mental health services utilization and costs, and work performance. Depress. Anxiety. 2010;27:78–89. doi: 10.1002/da.20580. [DOI] [PubMed] [Google Scholar]
  16. Borkovec T.D., Nau S.D. Credibility of analogue therapy rationales. J. Behav. Ther. Exp. Psychiatry. 1972;3:257–260. [Google Scholar]
  17. Boß L., Lehr D., Berking M., Riper H., Schaub M., Ebert D. Evaluating the (cost-)effectiveness of guided and unguided internet-based self-help for problematic alcohol use in employees - a three arm randomized controlled trial. BMC Public Health. 2015;15:1043. doi: 10.1186/s12889-015-2375-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Boß L., Lehr D., Reis D., Vis C., Riper H., Berking M., Ebert D.D. Reliability and validity of assessing user satisfaction with web-based health interventions. J. Med. Internet Res. 2016;18 doi: 10.2196/jmir.5952. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Bradley K.A., Debenedetti A.F., Volk R.J., Williams E.C., Frank D., Kivlahan D.R. AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol. Clin. Exp. Res. 2007;31:1208–1217. doi: 10.1111/j.1530-0277.2007.00403.x. [DOI] [PubMed] [Google Scholar]
  20. Brenes G.A. Anxiety, depression, and quality of life in primary care patients. Prim. Care Companion J. Clin. Psychiatry. 2007;9:437–443. doi: 10.4088/pcc.v09n0606. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Brown T.A., Barlow D.H. Comorbidity among anxiety disorders: implications for treatment and DSM-IV. J. Consult. Clin. Psychol. 1992;60:835–844. doi: 10.1037//0022-006x.60.6.835. (doi:0022-006X/92/S3.00) [DOI] [PubMed] [Google Scholar]
  22. Buntrock C., Ebert D.D., Lehr D., Cuijpers P., Riper H., Smit F., Berking M. Evaluating the efficacy and cost-effectiveness of web-based indicated prevention of major depression: design of a randomised controlled trial. BMC Psychiatry. 2014;14:1–10. doi: 10.1186/1471-244X-14-25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Buntrock C., Ebert D.D., Lehr D., Riper H., Smit F., Cuijpers P., Berking M. Effectiveness of a web-based cognitive behavioural intervention for subthreshold depression: pragmatic randomised controlled trial. Psychother. Psychosom. 2015;84:348–358. doi: 10.1159/000438673. [DOI] [PubMed] [Google Scholar]
  24. Buntrock C., Ebert D.D., Lehr D., Smit F., Riper H., Berking M., Cuijpers P. Effect of a web-based guided self-help intervention for prevention of major depression in adults with subthreshold depression a randomized clinical trial. JAMA. 2016:315. doi: 10.1001/jama.2016.4326. [DOI] [PubMed] [Google Scholar]
  25. Buntrock C., Ebert D.D., Lehr D., Smit F., Riper H., Berking M., Cuijpers P. Effect of a web-based guided self-help intervention for prevention of major depression in adults with subthreshold depression. JAMA. 2016;315:1854. doi: 10.1001/jama.2016.4326. [DOI] [PubMed] [Google Scholar]
  26. Buntrock C., Berking M., Smit F., Lehr D., Nobis S., Riper H., Cuijpers P., Ebert D. Preventing depression in adults with subthreshold depression: health-economic evaluation alongside a pragmatic randomized controlled trial of a web-based intervention. J. Med. Internet Res. 2017;19 doi: 10.2196/jmir.6587. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Bush K., Kivlahan D.R., McDonell M.B., Fihn S.D., Bradley K.A. The AUDIT alcohol consumption questions (AUDIT-C) Arch. Interal Med. 1998;158:1789–1795. doi: 10.1001/archinte.158.16.1789. [DOI] [PubMed] [Google Scholar]
  28. Buysse D.J., Reynolds C.F., Monk T.H., Berman S.R., Kupfer D.J. The Pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28:193–213. doi: 10.1016/0165-1781(89)90047-4. [DOI] [PubMed] [Google Scholar]
  29. Carey K.B., Neal D.J., Collins S.E. A psychometric analysis of the self-regulation questionnaire. Addict. Behav. 2004;29:253–260. doi: 10.1016/j.addbeh.2003.08.001. [DOI] [PubMed] [Google Scholar]
  30. Cheng S.K., Dizon J. Computerised cognitive behavioural therapy for insomnia: a systematic review and meta-analysis. Psychother. Psychosom. 2012;81:206–216. doi: 10.1159/000335379. [DOI] [PubMed] [Google Scholar]
  31. Chisholm D., Sanderson K., Ayuso-Mateos J.L., Saxena S. Reducing the global burden of depression: population-level analysis of intervention cost-effectiveness in 14 world regions. Br. J. Psychiatry. 2004;184:393–403. doi: 10.1192/bjp.184.5.393. [DOI] [PubMed] [Google Scholar]
  32. Christensen H., Batterham P., Mackinnon A., Griffiths K.M., Kalia Hehir K., Kenardy J., Gosling J., Bennett K. Prevention of generalized anxiety disorder using a web intervention, iChill: randomized controlled trial. J. Med. Internet Res. 2014;16 doi: 10.2196/jmir.3507. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Clarke G.N., Hawkins W., Murphy M., Sheeber L.B., Lewinsohn P.M., Seeley J.R. Targeted prevention of unipolar depressive disorder in an at-risk sample of high school adolescents: a randomized trial of a group cognitive intervention. J. Am. Acad. Child Adolesc. Psychiatry. 1995;34:312–321. doi: 10.1097/00004583-199503000-00016. [DOI] [PubMed] [Google Scholar]
  34. Cohen J. Academic Press; 1977. Statistical Power Analysis for the Behavioral Sciences. [Google Scholar]
  35. Connor K.M., Davidson J.R.T. Development of a new resilience scale: the Connor-Davidson resilience scale (CD-RISC) Depress. Anxiety. 2003;18:76–82. doi: 10.1002/da.10113. [DOI] [PubMed] [Google Scholar]
  36. Cuijpers P., van Straten A., Andersson G., van Oppen P. Psychotherapy for depression in adults: WWWa meta-analysis of comparative outcome studies. J. Consult. Clin. Psychol. 2008;76:909–922. doi: 10.1037/a0013075. (doi:2008-16943-011[pii]\r10.1037/a0013075) [DOI] [PubMed] [Google Scholar]
  37. Cuijpers P., Marks I.M., van Straten A., Cavanagh K., Gega L., Andersson G. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn. Behav. Ther. 2009;38:66–82. doi: 10.1080/16506070802694776. [DOI] [PubMed] [Google Scholar]
  38. Cuijpers P., Koole S.L., Van Dijke A., Roca M., Li J., Reynolds C.F. Psychotherapy for subclinical depression: meta-analysis. Br. J. Psychiatry. 2014;205:268–274. doi: 10.1192/bjp.bp.113.138784. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Cuijpers P., Sijbrandij M., Koole S.L., Andersson G., Beekman A.T., Reynolds C.F. Adding psychotherapy to antidepressant medication in depression and anxiety disorders: a meta-analysis. World Psychiatry. 2014;13:56–67. doi: 10.1002/wps.20089. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. de Graaf R., Bijl R.V., Smit F., Vollebergh W.A.M., Spijker J. Risk factors for 12-month comorbidity of mood, anxiety, and substance use disorders: findings from the Netherlands mental health survey and incidence study. Am. J. Psychiatry. 2002;159:620–629. doi: 10.1176/appi.ajp.159.4.620. [DOI] [PubMed] [Google Scholar]
  41. de Jonge P., Hadj F.B., Boffa D., Zdrojewski C., Dorogi Y., So A., Ruiz J., Stiefel F. Prevention of major depression in complex medically ill patients: preliminary results from a randomized, controlled trial. Psychosomatics. 2009;50:227–233. doi: 10.1176/appi.psy.50.3.227. [DOI] [PubMed] [Google Scholar]
  42. de Wit M., Pouwer F., Gemke R.J.B.J., Delemarre-van de Waal H.A., Snoek F.J. Validation of the WHO-5 well-being index (WHO-5) in adolescents with type 1 diabetes. Diabetes Care. 2007;30:2003–2006. doi: 10.2337/dc07-0447. [DOI] [PubMed] [Google Scholar]
  43. Dear B.F., Staples L.G., Terides M.D., Fogliati V.J., Sheehan J., Johnston L., Kayrouz R., Dear R., McEvoy P.M., Titov N. Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for social anxiety disorder and comorbid disorders: a randomized controlled trial. J. Anxiety Disord. 2016;42:30–44. doi: 10.1016/j.janxdis.2016.05.004. [DOI] [PubMed] [Google Scholar]
  44. Devilly G.J., Borkovec T.D. Psychometric properties of the credibility/expectancy questionnaire. J. Behav. Ther. Exp. Psychiatry. 2000;31:73–86. doi: 10.1016/s0005-7916(00)00012-4. [DOI] [PubMed] [Google Scholar]
  45. Dölemeyer R., Tietjen A., Kersting A., Wagner B. Internet-based interventions for eating disorders in adults: a systematic review. BMC Psychiatry. 2013;13 doi: 10.1186/1471-244X-13-207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Dybek I. 2008. Screening-Verfahren zur Entdeckung alkoholbezogener Störungen in Allgemeinarztpraxen: Reliabilität und Validität des AUDIT. (AUDIT-C und LAST) [Google Scholar]
  47. Ebert D.D., Christ O., Berking M. Development and validation of a self-report instrument for the assessment of emotion-specific regulation skills (ERSQ-ES)|Entwicklung und validierung eines fragebogens zur emotionsspezifischen selbsteinsch?tzung emotionaler kompetenzen (SEK-ES) Diagnostica. 2013;59 [Google Scholar]
  48. Ebert D.D., Lehr D., Smit F., Zarski A.-C., Riper H., Heber E., Cuijpers P., Berking M. Efficacy and cost-effectiveness of minimal guided and unguided internet-based mobile supported stress-management in employees with occupational stress: a three-armed randomised controlled trial. BMC Public Health. 2014;14 doi: 10.1186/1471-2458-14-807. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Ebert D.D., Thiart H., Laferton J.A.C., Berking M., Riper H., Cuijpers P., Sieland B., Lehr D. Restoring depleted resources: efficacy and mechanisms of change of an internet-based unguided recovery training for better sleep and psychological detachment from work. Health Psychol. 2015;34 doi: 10.1037/hea0000277. [DOI] [PubMed] [Google Scholar]
  50. Ebert D.D., Zarski A.C., Christensen H., Stikkelbroek Y., Cuijpers P., Berking M., Riper H. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One. 2015;10:1–15. doi: 10.1371/journal.pone.0119895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Ebert D.D., Buntrock C., Cuijpers P. Online intervention for prevention of major depression association of Infection in early life and risk of developing type 1 diabetes. JAMA. 2016;316:23–24. doi: 10.1001/jama.2016.9586. [DOI] [PubMed] [Google Scholar]
  52. Ebert D.D., Buntrock C., Cuijpers P., K, van Z., P, C., C, B., C, B., H, B Online intervention for prevention of major depression —reply. JAMA. 2016;316:881. doi: 10.1001/jama.2016.9586. [DOI] [PubMed] [Google Scholar]
  53. Ebert D.D., Nobis S., Lehr D., Baumeister H., Riper H.M., Auerbach R.P., Snoek F., Cuijpers P., Berking M. The 6-month effectiveness of internet-based guided self-help for depression in adults with type 1 and 2 diabetes mellitus. Diabet. Med. 2016 doi: 10.1111/dme.13173. [DOI] [PubMed] [Google Scholar]
  54. Ebert D.D., Buntrock C., Lehr D., Smit F., Riper H., Baumeister H., Cuijpers P., Berking M. Effectiveness of web- and mobile-based treatment of subthreshold depression with adherence-focused guidance. A single-blind randomised controlled trial. Behav. Ther. 2017 doi: 10.1016/j.beth.2017.05.004. [DOI] [PubMed] [Google Scholar]
  55. Ebert D.D., Cuijpers P., Muñoz R.F., Baumeister H. Prevention of mental health disorders using internet and mobile-based interventions: a narrative review and recommendations for future research. Front. Psychiatry. 2017;8 doi: 10.3389/fpsyt.2017.00116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Emmelkamp P.M.G., David D., Beckers T., Muris P., Cuijpers P., Lutz W., Andersson G., Araya R., Banos Rivera R.M., Barkham M., Berking M., Berger T., Botella C., Carlbring P., Colom F., Essau C., Hermans D., Hofman S.G., Knappe S., Ollendick T.H., Raes F., Rief W., Riper H., Van der Oord S., Vervliet B. Advancing psychotherapy and evidencebased psychological interventions. Int. J. Methods Psychiatr. Res. 2014;16:58–91. doi: 10.1002/mpr.1411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Erbe D., Eichert H.-C., Rietz C., Ebert D. Interformat reliability of the patient health questionnaire: validation of the computerized version of the PHQ-9. Internet Interv. 2016;5 doi: 10.1016/j.invent.2016.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Fischer E.H., Farina A. Attitudes toward seeking professional psychologial help: a shortened form and considerations for research. J. Coll. Stud. Dev. 1995;36:368–373. [Google Scholar]
  59. Fry J.P., Neff R.A. Periodic prompts and reminders in health promotion and health behavior interventions: systematic review. J. Med. Internet Res. 2009;11(e16):1–15. doi: 10.2196/jmir.1138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Fuhr K., Hautzinger M., Krisch K., Berking M., Ebert D.D. Validation of the behavioral activation for depression scale (BADS) - psychometric properties of the long and short form. Compr. Psychiatry. 2016;66 doi: 10.1016/j.comppsych.2016.02.004. [DOI] [PubMed] [Google Scholar]
  61. Global Burden of Disease Study 2013, Vos T., Barber R.M., Bell B., Bertozzi-Villa A., Biryukov S. Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990–2013: a systematic analysis for the global burden of disease study 2013. Lancet. 2015;386:743–800. doi: 10.1016/S0140-6736(15)60692-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Grawe K. Hogrefe; 2004. Neuropsychotherapie. [Google Scholar]
  63. Greenberg P.E., Birnbaum H.G. The economic burden of depression in the US: societal and patient perspectives. Expert. Opin. Pharmacother. 2005;6:369–376. doi: 10.1517/14656566.6.3.369. [DOI] [PubMed] [Google Scholar]
  64. Hamilton M. The assessment of anxiey states by rating. Br. J. Med. Psychol. 1959;32:50–55. doi: 10.1111/j.2044-8341.1959.tb00467.x. [DOI] [PubMed] [Google Scholar]
  65. Hautzinger M., Bailer M., Hofmeister D., Keller F. Hogrefe-Verlag Göttingen; Boston, MA: 2012. Allgemeine Depressions Skala Manual. [Google Scholar]
  66. Hofmann S.G., Smits J.A.J. Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials. J. Clin. Psychiatry. 2008;69:621–632. doi: 10.4088/jcp.v69n0415. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Horvath A.O., Greenberg L.S. Development and validation of the working alliance inventory. J. Couns. Psychol. 1989;36:223–233. [Google Scholar]
  68. Imamura K., Kawakami N., Furukawa T.A., Matsuyama Y., Shimazu A., Umanodan R., Kawakami S., Kasai K. Does internet-based cognitive behavioral therapy (iCBT) prevent major depressive episode for workers? A 12-month follow-up of a randomized controlled trial. Psychol. Med. 2015;45:1907–1917. doi: 10.1017/S0033291714003006. [DOI] [PubMed] [Google Scholar]
  69. Jacobson N.S., Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J. Consult. Clin. Psychol. 1991;59:12–19. doi: 10.1037//0022-006x.59.1.12. [DOI] [PubMed] [Google Scholar]
  70. Johansson R., Andersson G. Internet-based psychological treatments for depression. Expert. Rev. Neurother. 2012;12:861–870. doi: 10.1586/ern.12.63. [DOI] [PubMed] [Google Scholar]
  71. Joling K.J., van Marwijk H.W.J., Smit F., van der Horst H.E., Scheltens P., van de Ven P.M., Mittelman M.S., van Hout H.P.J. Does a family meetings intervention prevent depression and anxiety in family caregivers of dementia patients? A randomized trial. PLoS One. 2012;7 doi: 10.1371/journal.pone.0030936. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Kanter J.W., Mulick P.S., Busch A.M., Berlin K.S., Martell C.R. Measurement Instrument Database for the Social Science. 2012. Behavioral activation for depression scale (BADS) (long and short form) [Google Scholar]
  73. Kessler R.C., McGonagle K.A., Zhao S., Nelson C.B., Hughes M., Eshleman S., Wittchen H.-U., Kendler K.S. Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States - results from the National Comorbidity Survey. Arch. Gen. Psychiatry. 1994;51:8–19. doi: 10.1001/archpsyc.1994.03950010008002. [DOI] [PubMed] [Google Scholar]
  74. Kessler R.C., Chiu W.T., Demler O., Walters E.E. Prevalence, severity, and comorbidity of twelve-month DSM-IV disorders in the national comorbidity survey replication (NCS-R) Arch. Gen. Psychiatry. 2005;62:617–627. doi: 10.1001/archpsyc.62.6.617. [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. King J.T., Tsevat J., Roberts M.S. Measuring preference-based quality of life using the Europol EQ-5D in patients with cerebral aneurysms. Neurosurgery. 2009;65:565–572. doi: 10.1227/01.NEU.0000350980.01519.D8. [DOI] [PubMed] [Google Scholar]
  76. Kleiboer A., Donker T., Seekles W., van Straten A., Riper H., Cuijpers P. A randomized controlled trial on the role of support in internet-based problem solving therapy for depression and anxiety. Behav. Res. Ther. 2015;72:63–71. doi: 10.1016/j.brat.2015.06.013. [DOI] [PubMed] [Google Scholar]
  77. Klein Hofmeijer-Sevink M., Batelaan N.M., Van Megen H.J.G.M., Penninx B.W., Cath D.C., Van Den Hout M.A., Van Balkom A.J.L.M. Clinical relevance of comorbidity in anxiety disorders: a report from the Netherlands study of depression and anxiety (NESDA) J. Affect. Disord. 2012;137:106–112. doi: 10.1016/j.jad.2011.12.008. [DOI] [PubMed] [Google Scholar]
  78. Königbauer J., Letsch J., Doebler P., Ebert D.D., Baumeister H. Internet- and mobile-based depression interventions for people with diagnosed depression: a systematic review and Meta-analysis. J. Affect. Disord. 2017;178:131–141. [Google Scholar]
  79. Kroenke K., Spitzer R.L., Williams J.B.W. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 2001;16:606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  80. Kuckartz U., Ebert T., Rädiker S., Stefer C. 2009. Evaluation Online: Internetgestützte Befragung in der Praxis. [Google Scholar]
  81. Ladwig I., Rief W., Nestoriuc Y. What are the risks and side effects pf psychotherapy? - development of an inventory for the assessment of negative effects of psychotherapy (INEP) Verhaltenstherapie. 2014;24:252–263. [Google Scholar]
  82. Lehr D., Hillert A., Schmitz E., Sosnowsky N. Screening depressiver Störungen mittels Allgemeiner Depressions-Skala (ADS-K) und State-Trait Depressions Scales (STDS-T). Eine vergleichende Evaluation von Cut-Off-Werten. Diagnostica. 2008;54:61–70. [Google Scholar]
  83. Löwe B., Kroenke K., Herzog W., Gräfe K. Measuring depression outcome with a brief self-report instrument: sensitivity to change of the patient health questionnaire (PHQ-9) J. Affect. Disord. 2004;81:61–66. doi: 10.1016/S0165-0327(03)00198-8. [DOI] [PubMed] [Google Scholar]
  84. Löwe B., Decker O., Müller S., Brähler E., Schellberg D., Herzog W., Herzberg P.Y. Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Med. Care. 2008;46:266–274. doi: 10.1097/MLR.0b013e318160d093. [DOI] [PubMed] [Google Scholar]
  85. Luo N., Chew L.H., Fong K.Y., Koh D.R., Ng S.C., Yoon K.H., Vasoo S., Li S.C., Thumboo J. A comparison of the EuroQol-5D and the health utilities index mark 3 in patients with rheumatic disease. J. Rheumatol. 2003;30:2268–2274. (doi:0315162X-30-2268 [pii) [PubMed] [Google Scholar]
  86. Lyketsos C.G., Nestadt G., Cwi J., Heithoff K. The Life Chart Interview: a standardized method to describe the course of psychopathology. Int. J. Methods Psychiatr. Res. 1994 [Google Scholar]
  87. Martell C.R., Addis M.E., Jacobson N.S. W W Norton & Co; 2001. Depression in Context: Strategies for Guided Action. [Google Scholar]
  88. Martin A., Rief W., Klaiberg A., Braehler E. Validity of the brief patient health questionnaire mood scale (PHQ-9) in the general population. Gen. Hosp. Psychiatry. 2006;28:71–77. doi: 10.1016/j.genhosppsych.2005.07.003. [DOI] [PubMed] [Google Scholar]
  89. Matsubara C., Green J., Astorga L.T., Daya E.L., Jervoso H.C., Gonzaga E.M., Jimba M. Reliability tests and validation tests of the client satisfaction questionnaire (CSQ-8) as an index of satisfaction with childbirth-related care among Filipino women. BMC Pregnancy Childbirth. 2013;13(235):1–9. doi: 10.1186/1471-2393-13-235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  90. Mayo-Wilson E., Montgomery P. Media-delivered cognitive behavioural therapy and behavioural therapy (self-help) for anxiety disorders in adults (Review) Cochrane Database Syst. Rev. 2013:1–498. doi: 10.1002/14651858.CD005330.pub4. [DOI] [PubMed] [Google Scholar]
  91. Mayring P. Handbuch Qualitative Forschung in Der Psychologie. 2010. Qualitative inhaltsanalyse; pp. 601–613. [Google Scholar]
  92. Meyer T.J., Miller M.L., Metzger R.L., Borkovec T.D. Development and validation of the Penn state worry questionnaire. Behav. Res. Ther. 1990;28:487–495. doi: 10.1016/0005-7967(90)90135-6. [DOI] [PubMed] [Google Scholar]
  93. Mira A., Bretón-López J., García-Palacios A., Quero S., Baños R.M., Botella C. An internet-based program for depressive symptoms using human and automated support: a randomized controlled trial. Neuropsychiatr. Dis. Treat. 2017;13:987–1006. doi: 10.2147/NDT.S130994. [DOI] [PMC free article] [PubMed] [Google Scholar]
  94. Mondal P., Gjevre J.A., Taylor-Gjevre R.M., Lim H.J. Relationship between the Pittsburgh sleep quality index and the Epworth sleepiness scale in a sleep laboratory referral population. Nat. Sci. Sleep. 2013;5:15–21. doi: 10.2147/NSS.S40608. [DOI] [PMC free article] [PubMed] [Google Scholar]
  95. Nobis S., Lehr D., Ebert D.D., Baumeister H., Snoek F., Riper H., Berking M. Efficacy of a web-based intervention with mobile phone support in treating depressive symptoms in adults with type 1 and type 2 diabetes: a randomized controlled trial. Diabetes Care. 2015 doi: 10.2337/dc14-1728. [DOI] [PubMed] [Google Scholar]
  96. Olthuis J.V., Watt M.C., Bailey K., Hayden J.A. Therapist-supported internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst. Rev. 2015;3 doi: 10.1002/14651858.CD011565. [DOI] [PubMed] [Google Scholar]
  97. Păsărelu C.R., Andersson G., Bergman Nordgren L., Dobrean A. Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials. Cogn. Behav. Ther. 2017;46:1–28. doi: 10.1080/16506073.2016.1231219. [DOI] [PubMed] [Google Scholar]
  98. Radloff L.S. The CES-D scale: a self-report depression scale for research in the general population. Appl. Psychol. Meas. 1977;1:385–401. [Google Scholar]
  99. Rammstedt B., John O.P. Measuring personality in one minute or less: a 10-item short version of the big five inventory in English and German. J. Res. Pers. 2007;41:203–212. [Google Scholar]
  100. Reilly T.J., MacGillivray S.A., Reid I.C., Cameron I.M. Psychometric properties of the 16-item quick inventory of depressive symptomatology: a systematic review and meta-analysis. J. Psychiatr. Res. 2015;60:132–140. doi: 10.1016/j.jpsychires.2014.09.008. [DOI] [PubMed] [Google Scholar]
  101. Rener-Sitar K., John M.T., Bandyopadhyay D., Howell M.J., Schiffman E.L. Exploration of dimensionality and psychometric properties of the Pittsburgh sleep quality index in cases with temporomandibular disorders. Health Qual. Life Outcomes. 2014;12(1):10. doi: 10.1186/1477-7525-12-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  102. Richards D., Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin. Psychol. Rev. 2012;32:329–342. doi: 10.1016/j.cpr.2012.02.004. [DOI] [PubMed] [Google Scholar]
  103. Richardson K.M., Rothstein H.R. Effects of occupational stress management intervention programs: a meta-analysis. J. Occup. Health Psychol. 2008;13:69–93. doi: 10.1037/1076-8998.13.1.69. [DOI] [PubMed] [Google Scholar]
  104. Richardson J., Iezzi A., Khan M., Maxwell A. Validity and reliability of the assessment of quality of life (AQoL)-8D multi-attribute utility instrument. Patient. 2014;7:85–96. doi: 10.1007/s40271-013-0036-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  105. Riper H., Blankers M., Hadiwijaya H., Cunningham J., Clarke S., Wiers R., Ebert D., Cuijpers P. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One. 2014;9 doi: 10.1371/journal.pone.0099912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  106. Rosenberg M. Society and the adolescent self-image. Science. 1965;148(80):804. [Google Scholar]
  107. Rozental A., Boettcher J., Andersson G., Schmidt B., Carlbring P. Negative effects of internet interventions: a qualitative content analysis of patients' experiences with treatments delivered online. Cogn. Behav. Ther. 2015;44:223–236. doi: 10.1080/16506073.2015.1008033. [DOI] [PubMed] [Google Scholar]
  108. Rush A.J., Trivedi M.H., Ibrahim H.M., Carmody T.J., Arnow B., Klein D.N., Markowitz J.C., Ninan P.T., Kornstein S., Manber R., Thase M.E., Kocsis J.H., Keller M.B. The 16-item quick inventory of depressive. Biol. Psychiatry. 2003;54:573–583. doi: 10.1016/s0006-3223(02)01866-8. [DOI] [PubMed] [Google Scholar]
  109. Ryan R.M., Plant R.W., O'Malley S. Initial motivations for alcohol treatment: relations with patient characteristics, treatment involvement, and dropout. Addict. Behav. 1995;20:279–297. doi: 10.1016/0306-4603(94)00072-7. [DOI] [PubMed] [Google Scholar]
  110. Saarni S.I., Suvisaari J., Sintonen H., Pirkola S., Koskinen S., Aromaa A., Lönnqvist J. Impact of psychiatric disorders on health-related quality of life: general population survey. Br. J. Psychiatry. 2007;190:326–332. doi: 10.1192/bjp.bp.106.025106. [DOI] [PubMed] [Google Scholar]
  111. Shear M.K., Bilt J. Vander, Rucci P., Endicott J., Lydiard B., Otto M.W., Pollack M.H., Chandler L., Williams J., Ali A., Frank D.M. Reliability and validity of a structured interview guide for the Hamilton anxiety rating scale (SIGH-A) Depress. Anxiety. 2001;13:166–178. [PubMed] [Google Scholar]
  112. Sheehan D., Janvas J., Baker R., Harnett-Sheehan K., Knapp E., Sheehan M. 2006. Mini International Neuropsychiatric Interview. [Google Scholar]
  113. Sheffield J.K., Spence S.H., Rapee R.M., Kowalenko N., Wignall A., Davis A., McLoone J. Evaluation of universal, indicated, and combined cognitive-behavioral approaches to the prevention of depression among adolescents. J. Consult. Clin. Psychol. 2006;74:66–79. doi: 10.1037/0022-006X.74.1.66. [DOI] [PubMed] [Google Scholar]
  114. Spek V., Cuijpers P., Nyklicek I., Riper H., Keyzer J., Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol. Med. 2007;37:319–328. doi: 10.1017/S0033291706008944. [DOI] [PubMed] [Google Scholar]
  115. Spitzer R.L., Kroenke K., Williams J.B.W., Löwe B. A brief measure for assessing generalized anxiety disorder. Arch. Intern. Med. 2006;166:1092–1097. doi: 10.1001/archinte.166.10.1092. [DOI] [PubMed] [Google Scholar]
  116. Surgenor L.J. Attitudes towards seeking professional psychological help. Aust. J. Psychol. 1985;14:27–33. [Google Scholar]
  117. Thiart H., Lehr D., Ebert D.D., Berking M., Riper H. Log in and breathe out: internet-based recovery training for sleepless employees with work-related strain – results of a randomized controlled trial. Scand. J. Work Environ. Health. 2015;41:164–174. doi: 10.5271/sjweh.3478. [DOI] [PubMed] [Google Scholar]
  118. Thiart H., Ebert D.D., Lehr D., Nobis S., Buntrock C., Berking M., Smit F., Riper H. Internet-based cognitive behavioral therapy for insomnia: a health economic evaluation. Sleep. 2016;39 doi: 10.5665/sleep.6152. [DOI] [PMC free article] [PubMed] [Google Scholar]
  119. Thompson N.J., Patel A.H., Selwa L.M., Stoll S.C., Begley C.E., Johnson E.K., Fraser R.T. Expanding the efficacy of project UPLIFT: distance delivery of mindfulness-based depression prevention to people with epilepsy. J. Consult. Clin. Psychol. 2015;83:304–313. doi: 10.1037/a0038404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  120. Titov N., Dear B.F., Schwencke G., Andrews G., Johnston L., Craske M.G., McEvoy P. Transdiagnostic internet treatment for anxiety and depression: a randomised controlled trial. Behav. Res. Ther. 2011;49:441–452. doi: 10.1016/j.brat.2011.03.007. [DOI] [PubMed] [Google Scholar]
  121. Titov N., Dear B.F., Johnston L., Lorian C., Zou J., Wootton B., Spence J., McEvoy P.M., Rapee R.M. Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: randomised controlled trial. PLoS One. 2013;8 doi: 10.1371/journal.pone.0062873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  122. van Zoonen K., Buntrock C., Ebert D.D., Smit F., Reynolds C.F., Beekman A.T.F., Cuijpers P. Preventing the onset of major depressive disorder: a meta-analytic review of psychological interventions. Int. J. Epidemiol. 2014;43:318–329. doi: 10.1093/ije/dyt175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  123. van't Veer-Tazelaar P.J., van Marwijk H.W.J., van Oppen P., van Hout H.P.J., van der Horst H.E., Cuijpers P., Smit F., Beekman A.T.F. Stepped-care prevention of anxiety and depression in late life: a randomized controlled trial. Arch. Gen. Psychiatry. 2009;66:297–304. doi: 10.1001/archgenpsychiatry.2008.555. [DOI] [PubMed] [Google Scholar]
  124. van't Veer-Tazelaar P.J., van Marwijk H.W.J., van Oppen P., van Hout H.P.J., van der Horst H.E., Cuijpers P., Smit F., Beekman A.T.F. Stepped-care prevention of anxiety and depression in late life. Arch. Gen. Psychiatry. 2009;66:297–304. doi: 10.1001/archgenpsychiatry.2008.555. [DOI] [PubMed] [Google Scholar]
  125. Wang P.S., Lane M., Olfson M., Pincus H.A., Wells K.B., Kessler R.C. Twelve-month use of mental health services in the United States. Arch. Gen. Psychiatry. 2009;62:629–640. doi: 10.1001/archpsyc.62.6.629. [DOI] [PubMed] [Google Scholar]
  126. Willemse G.R.W.M., Smit F., Cuijpers P., Tiemens B.G. Minimal-contact psychotherapy for sub-threshold depression in primary care: randomised trial. Br. J. Psychiatry. 2004;185:416–421. doi: 10.1192/bjp.185.5.416. [DOI] [PubMed] [Google Scholar]
  127. World Health Organisation Wellbeing measures in primary health care/the depcare project. Rep. a WHO Meet. 1998;45 [Google Scholar]
  128. World Health Organization . 2008. The Global Burden of Disease: 2004 Update. [Google Scholar]
  129. Zalta A.K. A meta-analysis of anxiety symptom prevention with cognitive-behavioral interventions. J. Anxiety Disord. 2011;25:749–760. doi: 10.1016/j.janxdis.2011.02.007. [DOI] [PubMed] [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES