Skip to main content
NCBI home page
Internet Interventions logoLink to Internet Interventions
. 2022 Dec 12;31:100596. doi: 10.1016/j.invent.2022.100596

Internet-based emotion-regulation training added to CBT in adolescents with depressive and anxiety disorders: A pilot randomized controlled trial to examine feasibility, acceptability, and preliminary effectiveness

Marike A Wisman a,b,c,⁎, Julie Emmelkamp a,b,c, Jack JM Dekker b,d, Carolien Christ b,e
PMCID: PMC9760653  PMID: 36545446

Abstract

Background

Dysfunctional emotion regulation (ER) is associated with symptoms of depression and anxiety in adolescents. This pilot study aimed to examine the acceptability and feasibility of a guided internet-based emotion regulation training (ERT) added to cognitive behavioral therapy (CBT). Furthermore, we aimed to examine the feasibility of the randomized study design and to provide a first estimate of the effectiveness of CBT + ERT compared with CBT alone in adolescents with depressive or anxiety disorders.

Methods

In a pilot randomized controlled trial (RCT) with a parallel group design, 39 patients (13–18 years) with depressive or anxiety disorder were assigned to CBT + ERT (n = 21) or CBT (n = 18). Assessments at baseline, three-months and six-months follow-up included treatment adherence, satisfaction, depressive symptoms, anxiety symptoms, and ER strategies.

Results

Adherence to ERT was 66.5 %, and treatment satisfaction was adequate. 76.5 % of eligible patients participated in the study. Linear mixed-model analyses showed significantly reduced anxiety symptoms (p = .003), depressive symptoms (p = .017), and maladaptive ER (p = .014), and enhanced adaptive ER (p = .008) at six months follow-up in the CBT + ERT group compared to controls.

Limitations

The sample size was small, and results regarding effectiveness remain preliminary. Data-collection took place during COVID-19, which may have influenced the results.

Conclusions

Both the intervention and the study design were found to be feasible. In a larger RCT, however, improvement of recruitment strategy is necessary. Preliminary results indicate potential effectiveness in decreasing anxiety, depression, and emotion dysregulation in adolescents. The next step should be the development of an improved internet-based ERT and its evaluation in a larger RCT.

Trial registration

Registered on January 14th, 2020 in The Netherlands Trial Register (NL8304).

Keywords: Internet-based, Emotion regulation, Adolescence, Depression, Anxiety, eHealth

Highlights

  • •

    Internet-based emotion regulation training added to CBT was acceptable and feasible.

  • •

    The online emotion regulation training enhanced emotion regulation in adolescents.

  • •

    Preliminary results showed large effect sizes in decreasing depression and anxiety.

  • •

    Adding online emotion regulation training may help improve CBT outcome in youths.

1. Introduction

Anxiety and depressive disorders are common in children and adolescents (Merikangas et al., 2009; Polanczyk et al., 2015). Among youths, global prevalence rates are estimated at 6.5 % for anxiety disorders and 2.6 % for depressive disorders (Polanczyk et al., 2015). Comorbidity of these emotional disorders in adolescents is high (Costello et al., 2003; Garber and Weersing, 2010). Anxiety and depressive disorders negatively impact adolescents' social functioning, educational achievements, physical health, and quality of life (Essau et al., 2000; Jaycox et al., 2009). In addition, childhood anxiety and depressive disorders predict a range of mental health problems in adulthood (Essau et al., 2014; Johnson et al., 2018), and are associated with an increased risk of suicide, which is the fourth most prevalent cause of death in adolescents worldwide (WHO, 2021).

Adolescents with depressive or anxiety disorders are mainly treated with cognitive behavioral therapy (CBT), a widely-used treatment with medium-to-large effect sizes in this population (Crowe and McKay, 2017; Kendall and Peterman, 2015; James et al., 2020; Klein et al., 2007). CBT may be effectively delivered face-to-face or via the Internet, with comparable effects (Christ et al., 2020; Ebert et al., 2015). However, classic CBT protocols generally aim at either depression or anxiety. Adolescents with an anxiety disorder and comorbid depression show slower treatment response to anxiety-specific CBT (Berman et al., 2000; Suveg et al., 2009). In addition, evidence for the longitudinal benefits of CBT in young people remains scarce. Reviews by James et al. (2020) and Watanabe et al. (2007) found that disorder-specific CBT effectively reduces symptoms of anxiety and depression compared to waitlist at post-treatment, but there is no evidence supporting longer-term benefits. Given the high prevalence, high comorbidity and detrimental effects of anxiety and depressive disorders in adolescents, it is of utmost importance to identify common mechanisms underlying these disorders and optimize interventions in this vulnerable group.

Adolescence is characterized by growing autonomy, challenging life tasks (e.g., academic challenges), and major neurobiological, endocrinological and socio-emotional changes (Blakemore, 2008; Casey et al., 2008). Due to these changes, young people experience increasingly intense and unstable emotions and more frequent negative emotions than children (Ahmed et al., 2015; Zeman et al., 2006). The adaptive regulation of these emotions is important for reducing negative affect and psychological well-being (Aldao et al., 2010; Horn et al., 2011). However, evidence suggests that adolescents use more maladaptive and less adaptive emotion regulation strategies than children (Cracco et al., 2017). Emotion regulation (ER) refers to ‘the extrinsic and intrinsic processes responsible for monitoring, evaluating, and modifying emotional reactions, especially their intensive and temporal features, to accomplish one's goals’ (Thompson, 1994, pp. 27–28). ER is considered an important common underlying factor and may play a role in the etiology and maintenance of depressive and anxiety disorders (e.g., Aldao et al., 2010; Compas et al., 2017; Seager et al., 2014; Young et al., 2019). Maladaptive ER strategies (e.g., avoidance and rumination) are associated with more symptoms of depression and anxiety, whereas adaptive ER strategies (e.g., acceptance and problem solving) are linked to less symptoms (Aldao et al., 2010; Schäfer et al., 2016). Hence, ER appears an important transdiagnostic factor, and enhancing adolescent patients' ER skills in treatment may help decrease symptoms of depression and anxiety (Sloan et al., 2017). However, CBT only indirectly targets ER based processes, as it mainly targets dysfunctional cognitions and behavior.

In more recent third-wave psychological interventions, there is a growing interest in focusing directly on ER. Research has shown that integrating ER within a transdiagnostic framework combined with core elements of evidence-based interventions is promising in adult and adolescent samples with depressive or anxiety disorders (Eadeh et al., 2021; Ehrenreich-May et al., 2017; Pasarelu et al., 2007; Sakiris and Berle, 2019; Sandín et al., 2020). Examples of these transdiagnostic approaches are The Unified Protocols for Transdiagnostic Treatment of Emotional Disorders in Adolescents (UP-A) (Sherman and Ehrenreich-May, 2020), Rational Emotive Behavioral Therapy (REBT) (Pasarelu et al., 2021), Emotion-Focused Cognitive Behavioral Therapy (ECBT) (Suveg et al., 2006), and affect-focused psychodynamic treatment (Lindqvist et al., 2020). To date, however, these interventions are not widely available to patients with depressive and anxiety disorders, as CBT remains the treatment of first choice. Expanding regular treatment with a brief transdiagnostic ER training that can be flexibly added to CBT provides a more feasible way to enhance ER skills without changing the CBT itself. To our knowledge, no research has been conducted on the combination of CBT and an add-on transdiagnostic ER skills training in adolescent patients with depressive and anxiety disorders. In depressed adults, the addition of an ER skills training to CBT significantly decreased depressive symptoms compared to CBT alone (Berking et al., 2013).

The Internet is ubiquitous in the lives of Dutch adolescents: 99.7 % of 12–25-year-olds have access to the Internet (Central Bureau for Statistics, 2019). Numerous meta-analyses have demonstrated that internet-delivered treatment for anxiety and depression provide a feasible and effective alternative to face-to-face treatment in youths (Christ et al., 2020; Ebert et al., 2015; Grist et al., 2019; Lehtimaki et al., 2021). Providing emotion regulation training (ERT) as an add-on program via the internet is expected to fit the lifestyle of adolescents and minimalize their time-consuming visits to a health care institution.

This pilot randomized controlled trial (RCT) aims to examine the acceptability and feasibility (i.e., system usability, treatment adherence, and treatment satisfaction in patients and therapists) of an internet-based transdiagnostic ERT added to CBT for adolescents aged 13–18 with depressive and anxiety disorders. In addition, the current study aims to determine whether an RCT on its effectiveness is feasible regarding i) recruitment rates, ii) patient acceptability, and iii) study adherence rates. Furthermore, this study provides a first estimate of the effectiveness of CBT + ERT, as compared with classic CBT alone, in reducing depressive symptoms, anxiety symptoms, and ER difficulties. We hypothesized that depressive symptoms, anxiety symptoms, and ER difficulties would decrease significantly more in CBT + ERT compared with CBT alone.

2. Methods

2.1. Study design

This study was a two-arm pilot randomized controlled trial with a parallel group design that compared the effects of CBT + ERT versus CBT alone. After baseline assessment, participants were randomly assigned to either CBT + ERT or CBT.

2.2. Participants

Participants were patients aged 13–18, who were diagnosed with depressive disorder or anxiety disorder and were enrolled for cognitive behavioral therapy at Arkin Jeugd & Gezin (translation: Youth & Family) in Amsterdam. Participants had to be at least moderately proficient in Dutch and had regular access to a computer, tablet or mobile phone with internet connection. Participants with comorbid diagnoses were included, except for those with acute suicidal behavior or a current psychotic disorder.

In total, 51 patients were screened for eligibility. 39 (76.5 %) participants were eligible for inclusion and completed the baseline assessment. Fig. 1 shows the participant flow throughout the study. From 64,1 % (n = 25) of the included participants, one parent provided observer-based data. 21 respondents were randomized to CBT + ERT and 18 respondents to CBT alone. Participants' age ranged between 14 and 18 years (M = 16.41, SD = 1.23), and 76.9 % were girls (n = 30). Of the total sample, 71.8 % (n = 28) were diagnosed with depressive disorder and 28.2 % (n = 11) with anxiety disorder as primary diagnosis. All respondents were born in The Netherlands; however, 46.2 % (n = 18) were second generation and had at least one parent born in Western or Non-Western foreign countries. No significant baseline differences were found regarding all clinical study variables. Table 1 presents the demographic characteristics and the results of the baseline tests of differences.

Fig. 1.

Fig. 1

Open in a new tab

Flowchart of recruitment, enrollment, and follow-up of participants.

Table 1.

Demographic characteristics.

Total sample
(N = 39)		 CBT
(n = 18)		 CBT + ERT
(n = 21)		 t/X2 p
Age in years, mean (SD) 16.4 (1.2) 16.3 (1.2) 16.5 (1.25) 0.36 .72
Sex, n (%) 0.01 .91
 Female 30 (76.9) 14 (77.8) 16 (76.2)
 Male 9 (23.1) 4 (22.2) 5 (23.8)
Current school level, n (%) 3.35 .65
 Lower general secondary 4 (10.25) 3 (16.7) 1 (4.8)
 Senior general secondary 12 (30.8) 6 (33.3) 6 (28.5)
 Pre-university 18 (46.1) 8 (44.4) 10 (47.6)
 Intermediate vocational 4 (10.25) 1 (5.6) 3 (14.3)
 University 1 (2.6) 0 (0.0) 1 (4.8)
Ethnicity, n (%) 6.33 .04
 Dutch 21 (53.8) 13 (72.2) 8 (38.1)
 2nd generation Western 6 (15.4) 3 (16.7) 3 (14.3)
 2nd generation Non-Western 12 (30.8) 2 (11.1) 10 (47.7)
Computer use, n (%) 0.11 .95
 Daily 28 (71.8) 13 (72.2) 15 (71.4)
 Weekly 5 (12.8) 2 (11.1) 3 (14.3)
 Less than once a week 6 (15.4) 3 (16.7) 3 (14.3)
Primary diagnosis, n (%) 2.07 .84
 Depressive disordera 28 (71.8) 12 (66.7) 16 (76.2)
 Anxiety disorderb 11 (28.2) 6 (33.3) 5 (23.8)
Comorbidity depression/anxiety, n (%) 9 (23.1) 4 (22.2) 5 (23.8)
Open in a new tab

Abbreviations: CBT: Cognitive Behavioral Therapy; ERT: Internet-based Emotion Regulation Therapy.

a

Persistent depressive disorder or depressive disorder.

b

Generalized anxiety disorder, panic disorder, social phobia, or anxiety not otherwise specified.

2.3. Procedure

2.3.1. Recruitment and consent

From October 2018 to April 2020, patients referred to three locations of Arkin Jeugd & Gezin were screened for eligibility by a clinician during the intake using the Mini International Neuropsychiatric Interview (MINI). All eligible patients who had agreed to be approached by a researcher were contacted by telephone after one week by a research assistant. All participants gave written informed consent. For patients aged below 16, parents had to provide written informed consent as well. Parents who were willing to complete questionnaires about their child signed an additional informed consent, although parental cooperation was not mandatory for participation of the child.

2.3.2. Data collection

Baseline assessment took place two weeks before the start of CBT. Follow-up assessments were administered at 3 months (T1) and 6 months (T2) after baseline assessment. Participants and cooperating parents received an email with a link to the self-report questionnaires, conducted via online program NetQuestionnaire. For each completed assessment, the participant and parent each received a voucher of 10 euros.

2.3.3. Randomization

After completion of the baseline assessment by the adolescent, randomization was carried out by an independent data-manager of the Arkin Research Department. The data-manager was not involved in the project, nor in providing mental health care. Randomization took place at an individual level, stratified by gender and treatment location using a computer-generated block randomization schedule, with a block size of 4. To ensure that an equal number of patients was allocated to CBT + ERT and CBT, the allocation ratio was 1:1. To prevent selection bias, researchers were blind to block size and order, and did not have access to the randomization schedule. Due to the nature of the treatments, blinding of participants, parents, and therapists to treatment condition was not applicable.

2.4. Intervention

2.4.1. Treatment as usual: CBT

All participants received classic CBT, a commonly used evidence-based therapy for anxiety and depression among adolescents (Buitelaar et al., 2009; Kendall and Peterman, 2015; Klein et al., 2007). CBT is characterized by 1) behavioral analyses (i.e., identifying maladaptive thinking patterns, emotional responses, and behaviors), 2) cognitive restructuring, and 3) behavioral activation, experiments, and exposure (Beck, 2005; Seligman and Ollendick, 2011). In this study, CBT was provided by experienced psychologists and consisted of individual sessions of 45 min and homework assignments in between the appointments. Every participant had one CBT therapist.

CBT was disorder-specific, and focused on either depressive symptoms or anxiety symptoms. The specific CBT protocol used, was tailored to the main diagnosis and age of the participant.

2.4.2. ERT

Alongside CBT, participants in the experimental condition engaged in an internet-based emotion regulation training (ERT) delivered via Therapieland (https://therapieland.nl): a widely-used e-mental health platform. Patients were guided by a psychologist trained in the application of internet-based ERT and providing online feedback. After each online session, the psychologist provided feedback using secured email within the online platform. ERT consisted of six online sessions and two appointments with the ERT therapist. The first face-to-face/screen-to-screen session was scheduled at the start of ERT, in order for patients to become acquainted with their psychologists, and vice versa, to get familiar with the eHealth platform, and to determine treatment goals. This session was planned approximately two weeks after the start of CBT. In addition, after the first half of ERT, a second face-to-face/screen-to-screen session was scheduled to monitor progress, evaluate treatment goals, discuss motivation and adolescents' opinion about the training. ERT took place simultaneously with CBT. The program included various methods such as videos with psycho-education, exercises, and a library with relevant videos. Based on the patient's preference, the six ERT sessions were completed on a weekly or fortnightly schedule. Parents did not participate in the ERT. Text Box 1 provides an overview of the intervention. For their ERT sessions, all patients were guided by a therapist other than their CBT therapist. Because of the COVID pandemic, some of the face-to-face sessions of both ERT and CBT were converted to screen-to-screen sessions during this study.

Box 1. Session overview.

Session 1

This session contains a short introduction and summary of the program and explains the functionalities of the online platform. Adolescents establish and score treatment goals on a 1–7 range with regard to ER.

Session 2

This session starts with a video about different emotions and the origin, function, and characteristics of emotions. It focusses on the first core element: emotion comprehension, which refers to the knowledge to identify and understand others' emotions by bodily and facial cues, within specific social contexts (Harris et al., 2016). Through exercises, adolescents learn to identify emotions in others (i.e., distinction between emotions), learn to identify possible emotions in situations, and learn the expression of emotions in adaptive and maladaptive ways (e.g., facial expressions, and communication techniques).

Session 3

The third session starts with a video with psycho-education about emotional awareness (in line with Southam-Gerow, 2013). There is a shift from emotions in others to emotions in their selves. The link between emotions, situations, thoughts, behaviors, and consequences is discussed. Adolescents gain knowledge about their own emotions; characteristics and intensity of emotions (including emotion thermometers), intertwinement and influence of different emotions, helpful or unhelpful behavioral tendencies, and corresponding situations.

Session 4

This session focusses on adaptive emotion regulation strategies. The first part contains a video about accepting difficult emotions. The second part concerns gaining control of difficult emotions, expressing emotions in helpful ways, and learning behavioral strategies (e.g., sharing, writing, mindfulness, moving, listening to music) that help coping with painful emotions. These strategies are based on the functional model, described by Gross and Thompson (2007).

Session 5

This session focusses on lifestyle (sleep, food, sports, and relaxation), and the influence of lifestyle on emotion regulation. In addition, it contains a psycho-education video about the Window of Tolerance. This part of the program is based on the prevention skills training of Linehan (1993), and the connection between coping and lifestyle.

Session 6

The final session concerns evaluation and relapse prevention. Adolescents evaluate their treatment goals and set up a plan to use ER skills after finishing ERT. Furthermore, adolescents describe their pitfalls, helpful behavior, and support network.

Alt-text: Box 1

2.5. Outcome measures

2.5.1. Patient and therapist satisfaction

At T2, participants in the experimental condition filled in an adapted version of the Client Satisfaction Questionnaire (CSQ-8) about satisfaction with the online ERT. The Dutch translation of the CSQ was found reliable and valid in mental health outpatients in The Netherlands (De Brey, 1983). The total score of the questionnaire ranges from 8 to 32, with a higher score indicating a higher satisfaction. In the current study, the CSQ showed good internal consistency (Cronbach's α = 0.88). ERT therapists completed an adapted version of the 3-item Client Satisfaction Questionnaire (CSQ-3), and filled in the System Usability Scale (SUS), a 10-item validated questionnaire evaluating usability of internet-based interventions as perceived by therapists. The total score ranges from 0 to 100. A score above 68 represents acceptable usability (Sauro and Lewis, 2016). Lastly, online ERT adherence was operationalized as the percentage of the online ERT program followed.

2.5.2. RCT feasilibity

The RCT design was considered feasible if at least 75 % of eligible patients and 50 % of eligible parents agreed to participate in the study. Furthermore, study adherence had to be at least 75 % and is defined as the percentage of participants who completed T2 assessments. See Table 2 for the schedule of assessments.

Table 2.

Schedule of assessments.

Variable Instrument T0 (baseline) T1 (3 months follow-up) T2 (6 months follow-up)
Participant
Demographics X
Emotion regulation FEEL-KJ X X X
Depressive symptoms CDI-2 X X X
Anxiety symptoms SCARED-NL X X X
Internalizing problems YSR-int X X
Client satisfactiona CSQ-8 X



Parent
Depressive symptoms CDI-2 parent version X X X
Anxiety symptoms SCARED-NL parent version X X X
Internalizing problems CBCL-int X X



Therapist
Severity of illness CGI X X X
Therapist satisfaction CSQ-3 X
System usability SUS X
Open in a new tab

Note: FEEL-KJ: Questionnaire to Survey Emotion Regulation in Children and Adolescents, Dutch version; CDI-2: Children's Depression Inventory; SCARED-NL: Screen for Child Anxiety Related Emotional Disorders, Dutch version; YSR-int: Youth Self Report, internalizing scale; CSQ-8: Client Satisfaction Questionnaire; CBCL-int: Child Behavior Checklist, internalizing scale; CGI: Clinical Global Impression Scale; CSQ-3: Client Satisfaction Questionnaire, adapted for therapists; SUS: System Usability Scale.

a

Only the CBT + ERT group filled in client satisfaction.

2.5.3. Depressive symptoms

Self-reported depressive symptoms were measured by the Children's Depression Inventory ([CDI-2]; Bodden et al., 2016; Craighead et al., 1998). The CDI-2 is a revision of the CDI (Kovacs, 1992) and consists of 28 items, rated on a three-point Likert scale. In addition, parents reported the depressive symptoms of their children on the CDI-2 parent version (Bodden et al., 2016; Craighead et al., 1998), which consists of 17 items, rated on a three-point Likert scale. The total score of the questionnaires was used, with a higher score indicating a higher level of depressive symptoms. The CDI-2 was found to be a reliable and valid instrument among general and clinical populations (Bae, 2012; Bodden et al., 2016). In this study, the self-report and parent-report scales demonstrated good internal consistency, with Cronbach's α = 0.87 and 0.86, respectively.

2.5.4. Anxiety symptoms

Self-reported anxiety symptoms were measured by the Dutch version of the Screen for Child Anxiety Related Emotional Disorders ([SCARED-NL]; Muris et al., 2007). This is a 69-item questionnaire measuring severity of DSM-IV anxiety disorders. All items are rated on a three-point Likert scale. In addition, children's parent-reported anxiety symptoms were measured by the 69-item SCARED-NL parent version (Muris et al., 2007). The total scores of the questionnaires were used, with a higher score indicating a higher level of anxiety symptoms. The SCARED was found reliable and valid in outpatient children and their parents (Birmaher et al., 1997; Hale et al., 2011). In the current study, both the SCARED-NL self-report and parent-report version showed a high internal consistency (both 0.95).

2.5.5. Emotion regulation

ER was measured with the Dutch version of the Questionnaire to Survey Emotion Regulation in Children and Adolescents, which was originally developed in German ([FEEL-KJ]; Braet et al., 2013; Cracco et al., 2015; Grob and Smolenski, 2005). The questionnaire consists of 90 items and covers 15 different primary ER strategies. These strategies are divided in two secondary scales: Adaptive ER and Maladaptive ER. Adaptive ER includes seven primary strategies (Problem Solving, Distraction, Humor Enhancement, Acceptance, Forgetting, Cognitive Problem Solving, and Revaluation). Maladaptive ER includes five primary strategies (Giving up, Aggressive Reactions, Withdrawal, Self-devaluation, and Rumination). Furthermore, there are three other primary ER strategies (Social Support, Expression, and Emotional Control). Participants rate the items on a five-point Likert-scale, separately for the emotions anger, fear and sadness. In this study, the Adaptive ER scale and the Maladaptive ER scale were used. The total scores of the different strategies included in the scale were used. The FEEL-KJ was found to be a reliable and valid instrument in the general population (Braet et al., 2013; Cracco et al., 2015). In the current sample, the internal consistency of the FEEL-KJ Maladaptive and Adaptive scales were high (α = 0.93 and 0.92, resp.).

2.5.6. Internalizing problems

Self-reported internalizing problems were assessed by the Dutch version of the Youth Self Report ([YSR]; Achenbach and Rescorla, 2001; Verhulst and Van der Ende, 2013). The internalizing scale of the YSR consists of 25 items, which are scored on a 3-point Likert scale. In addition, parents reported adolescents' internalizing problems by the 26-item internalizing scale of the Child Behavior Checklist ([CBCL]; Achenbach and Rescorla, 2001; Verhulst and Van der Ende, 2013). Total scores were used, with a higher score indicating a higher level of internalizing problems. The internalizing scales of the YSR and CBCL were found reliable and valid in general and clinical populations (Verhulst and Van der Ende, 2013). In the current sample, the internal consistency of the YSR and CBCL internalizing scales were high, respectively 0.90 and 0.92.

2.5.7. Severity of illness

The severity of the patient's illness (CGI-S) and global improvement (CGI-I) were measured by the Clinical Global Impression Scale ([CGI]; Guy, 1976). CBT therapists filled in the CGI-S item at the start of treatment and both questions at three months, and at six months follow-up. The CGI-S assesses the severity of patient's illness on a 7-point Likert scale (1 = normal to 7 = severely ill). The CGI-I rates the clinical improvement at the moment of assessment compared to baseline, on a 7-point scale (1 = very much improved to 7 = very much worse). Results on validity of this questionnaire have been contradictory (e.g., Busner et al., 2009; De Beurs et al., 2019; Zaider et al., 2003). However, its use is widely accepted and the questionnaire is one of the most frequently used therapist-assessments in psychiatry (Forkmann et al., 2011).

2.6. Statistical analyses

The primary outcome measures were depressive symptoms (CDI-2) and anxiety symptoms (SCARED-NL) at 6 months follow-up (T2). Secondary outcomes included ER difficulties (FEEL-KJ Maladaptive ER and Adaptive ER), internalizing problems (YSR and CBCL), and severity of illness according to the therapist (CGI).

We checked if assumptions of linearity, homogeneity of variance, and normality were met. After inspecting the data, one outlier was found for the variable maladaptive ER. This outlier was winsorized (Wilcox, 2005). For the main study variables, observed means and estimated means were calculated. Descriptive statistics were used to calculate adherence rates and treatment satisfaction. Independent sample t-tests were used to determine whether treatment completers differed from non-completers regarding depressive symptoms, anxiety symptoms, maladaptive ER, and adaptive ER, and to examine the difference in total treatment time spent (in minutes) between both conditions.

To determine the effectiveness of CBT + ERT on primary outcomes and secondary outcome ER, linear mixed-model analyses were performed with a two-level structure (repeated measures, patients). First, within-group effect sizes were calculated for each condition by dividing the difference between post-test and pre-test means by the pre-test standard deviation. Second, we evaluated the overall treatment effect by fitting a model with treatment condition as fixed effect and the baseline value of the outcome variable as covariate. Third, we evaluated between-group differences at the separate follow-up time-points by adding time and an interaction between condition and time to the model. Time was treated as a categorical variable, represented by dummy variables. Fourth, effect sizes (Cohen's d) were computed from the multilevel estimates. Between-group effect sizes were established by dividing the estimated mean difference between the two groups by the pooled standard deviation. Effect sizes of 0.2 were considered small, 0.5 moderate and 0.8 large (Cohen, 1988). Because we used mixed-model analyses, imputation of missing data was not necessary (Twisk, 2013).

Two multiple regression analyses were conducted with self-reported and parent-reported internalizing problems at 6 months follow-up as outcome variables, and condition (dichotomous) and baseline level of the outcome variable as predictors. Because these variables were only assessed at baseline and follow-up, conducting linear mixed-model analyses was not possible.

A chi-square test was used to compare the proportion of treatment responders according to the therapist-reported CGI-I score between groups. In this analysis, patients with a CGI-I score of 1 or 2 at T2 were labeled as ‘responder-to-treatment’, and all others were labeled as ‘non-responder’. A t-test was used to analyze between-group-differences in the therapist-reported change of symptom severity, as measured by the change in CGI-S from T0 to T2.

Analyses of depressive symptoms, anxiety symptoms, and ER strategies were performed according to the intention-to-treat principle. Analyses of internalizing problems and therapist-rated severity of illness were performed in respondents who completed all assessments. All data were analyzed using SPSS version 26. All reported P-values are two-tailed and P-values of <0.05 were considered significant.

3. Results

3.1. Acceptability and feasibility

3.1.1. Intervention adherence

At T2, adherence to the online ERT intervention was 66.5 %, and the average number of logins was 11.24 (range 0–45; SD = 11.43). 9.5 % (2/21) of the respondents completed 0 % of ERT. 66.6 % (14/21) followed over half of the intervention, and 42.9 % (9/21) completed all online sessions. In participants who completed at least 1 session of the intervention (n = 17), the average adherence was 81.9 % and the average number of logins was 12.94 (range 2–45, SD = 11.88). Participants who completed at least 75 % of the intervention were considered treatment completers (n = 11). Completers had slightly lower baseline self-reported depressive symptoms (M = 23.27; SD = 6.08 vs. M = 25.00; SD = 7.30); t(19) = 0.59, p = .562, and anxiety symptoms (M = 53.64; SD = 25.28 vs. M = 61.80; SD = 19.69) compared to non-completers; t(19) = 0.82, p = .423, but differences were not significant. No differences in maladaptive ER strategies; t(19) = 0.38, p = .707, and adaptive ER strategies; t(19) = 0.10, p = .924, were found between completers and non-completers.

At six months post-baseline, no significant differences in total received treatment time (in minutes) between the CBT group (n = 17; M = 965.53; SD = 465.09; median = 915.00; IQR = 637.50) and CBT + ERT group (n = 21; M = 982.57; SD = 386.48; median = 1025.00; IQR = 592.50) were found (t(36) = −0.12; p = .902). In this analysis, only time spent on CBT (only face-to-face or screen-to-screen contact) and ERT (only face-to-face or screen-to-screen, and online feedback) were included. Minutes of all other treatments (e.g., pharmacotherapy sessions, group treatment) and psychodiagnostic assessment were excluded.

3.1.2. Treatment satisfaction

The mean total satisfaction score of participants (n = 13) was 19.00 (SD = 3.67; range 8–32), indicating a moderate satisfaction about the internet-based intervention.

The majority of participants (76.9 %) stated that the quality of internet-based help was good, and 61.5 % reported that they received the help they had hoped. Only 30.8 % would recommend the internet-based intervention to friends. Of the therapists, 100 % (n = 6) stated that ERT met all or most of their needs, and 83.3 % would use the online ERT in the future for their patients if possible. However, 66.7 % (n = 4) were slightly dissatisfied by the online ERT they guided. The mean SUS total score, reported by therapists, was 75.83 (SD = 7.53; range 67.5–85.0), which is above average and indicates sufficient system usability of the online ERT.

3.1.3. Study eligibility and adherence rates

The flow of participants through the study is presented in Fig. 1. 39 (76,5 %) of the 51 eligible respondents participated in the study. Ten respondents declined to participate in the study with various reasons: too time-consuming (n = 4), not helpful (n = 2), too difficult (n = 1), and no reason (n = 3). Two other respondents did not participate because of external reasons: one incomplete consent of parental authority, and one treatment termination because of financial reasons before signing informed consent. 79.5 % of the participants completed all assessments. Of 64.1 % of the included participants, a parent completed parent-reported questionnaires. Of the other 35.9 %, no parent or caregiver participated in the study.

3.2. Treatment effects

Table 3 illustrates observed and estimated means for the patient-rated outcomes per group at all timepoints. As can be seen in this table, the CBT group showed no amelioration at T2 on all main study variables, whereas the CBT + ERT group did. The CBT + ERT group showed a decrease in self-reported depressive symptoms (d = 1.12), anxiety symptoms (d = 0.53), internalizing symptoms (d = 0.73) and maladaptive ER (d = 0.58), and an increase in self-reported adaptive ER (d = 0.84).

Table 3.

Observed means and estimated means of main study variables in experimental (CBT + ERT) condition and control (CBT) condition.

Variable Assessment Observed means
 Estimated means
CBT + ERT
CBT
 CBT + ERT
 CBT
n M (SD) n M (SD) M (SD) M (SD)
CDI-2 T0 (baseline) 21 24.10 (6.58) 18 28.28 (10.10)
T1 (3 months) 18 21.61 (10.19) 18 26.33 (10.10) 23.83 (6.88) 24.19 (6.96)
T2 (6 months) 17 16.71 (10.52) 16 26.94 (9.55) 18.67 (6.79) 24.45 (6.80)
SCARED T0 (baseline) 21 57.52 (22.62) 18 61.94 (24.30)
T1 (3 months) 19 53.79 (24.96) 18 62.83 (26.47) 55.71 (13.57) 60.59 (13.58)
T2 (6 months) 16 45.44 (20.88) 16 64.06 (25.49) 47.84 (13.36) 62.41 (13.42)
FEEL-KJ Maladaptive ER T0 (baselines) 21 101.48 (15.93) 18 106.28 (21.88)
T1 (3 months) 17 93.71 (16.01) 18 105.67 (17.10) 96.93 (10.45) 102.71 (10.64)
T2 (6 months) 17 92.29 (19.25) 16 104.81 (17.15) 93.58 (10.42) 102.81 (10.44)
FEEL-KJ Adaptive ER T0 (baseline) 21 110.19 (22.59) 18 113.72 (23.31)
T1 (3 months) 17 122.06 (24.38) 18 121.833 (21.72) 123.49 (17.03) 120.22 (17.33)
T2 (6 months) 17 129.23 (24.77) 16 114.25 (28.52) 128.97 (17.02) 112.65 (17.03)
YSR T0 (baseline) 21 33.57 (11.24) 18 36.67 (11.01)
T2 (6 months) 17 25.42 (13.58) 16 36.00 (11.30)
Open in a new tab

Note: CBT: Cognitive Behavioral Therapy; ERT: Internet-based Emotion Regulation Therapy; CDI-2: Children's Depression Inventory; SCARED-NL: Screen for Child Anxiety Related Emotional Disorders, Dutch version; FEEL-KJ: Questionnaire to Survey Emotion Regulation in Children and Adolescents; ER: Emotion regulation; YSR: Youth Self Report; M: mean; SD: standard deviation.

The results of the linear mixed-model analyses are shown in Table 4. A significant time X group interaction was detected for self-reported depressive symptoms at 6-months follow-up (B = −5.88, 95 % CI = −10.68 to −1.08, p = .017), with the CBT + ERT group showing a larger symptom reduction than the CBT group. Hence, CBT + ERT was superior to CBT alone in decreasing depressive symptoms at 6-months follow-up, with a moderate effect size (Cohen's d = 0.70). Similarly, CBT + ERT was more effective than CBT alone in reducing self-reported anxiety symptoms at 6-months follow-up (Time x group interaction: B = −14.57, 95 % CI = −24.05 to −5.10, p = .003), with a moderate effect size (Cohen's d = 0.62). A similar intervention effect was found for maladaptive ER at T2, with a larger reduction of self-reported maladaptive ER strategies (B = −9.23, 95 % CI = −16.53 to −1.93, p = .014) and a larger increase of self-reported adaptive ER strategies (B = 16.32, 95 % CI = 4.43 to 28.20, p = .008) for the CBT + ERT group compared to the CBT group, with moderate effect sizes (Cohen's d = 0.49 and 0.71, respectively). Lastly, for parent-reported depressive symptoms and anxiety symptoms, no differences in treatment effects were found between both conditions.

Table 4.

Results of the linear mixed model analyses.

B p 95 % CI
 Cohen's d
Lower Upper
CDI-2 Overall −3.14 .147 −7.43 1.16
T1 −0.36 .879 −5.06 4.34 0.04
T2 −5.88 .017 −10.68 −1.08 0.70
SCARED-NL Overall −9.25 .024 −17.21 −1.28
T1 −4.88 .280 −13.83 4.07 0.21
T2 −14.57 .003 −24.05 −5.10 0.62
FEEL-KJ maladaptive ER Overall −7.54 .019 −13.78 −1.29
T1 −5.78 .113 −12.97 1.40 0.31
T2 −9.23 .014 −16.53 −1.93 0.49
FEEL-KJ adaptive ER Overall 9.56 .073 −0.92 20.04
T1 3.27 .576 −8.39 14.93 0.14
T2 16.32 .008 4.43 28.20 0.71
CDI-2 parent Overall −1.86 .386 −6.22 2.49
T1 −1.20 .624 −6.10 3.71 0.16
T2 −2.60 .301 −7.61 2.41 0.34
SCARED-NL parent Overall −0.80 .842 −9.03 7.43
T1 −1.15 .813 −10.90 8.59 0.05
T2 −0.30 .953 −10.41 9.82 0.01
Open in a new tab

Abbreviations: CDI-2: Children's Depression Inventory; SCARED-NL: Screen for Child Anxiety Related Emotional Disorders, Dutch version; FEEL-KJ: Fragebogen zur Erhebung der Emotionsregulation bei Kindern und Jugendlichen, Dutch version; ER: Emotion regulation.

Multiple regression analysis showed a significant regression equation of treatment condition and baseline severity on self-reported internalizing problems at T2, F(2, 30) = 14.53, p < .001, R2 = 0.49. CBT + ERT was more effective in reducing self-reported internalizing problems than CBT alone (B = 8.68; 95 % CI = 1.59 to 15.78; p = .018). A significant regression equation of treatment condition and baseline severity on parent-reported internalizing problems at T2 was found, F(2, 20) = 14.33, p < .001, R2 = 0.59. However, no differences were found between the treatment conditions (p = .780).

A chi-square test showed no significant difference in therapist-reported treatment response between CBT and CBT + ERT (X2(1) = 3.42, p = .065), although the CBT + ERT group showed a higher percentage of responders (57.9 %) compared to the CBT group (27.8 %). Lastly, the difference between the CBT group (M = −0.61; SD = 1.20) and CBT + ERT group (M = −1.42; SD = 1.87) in the therapist-reported change of CGI severity score between T0 and T2 was not significant (t(35) = 1.56; p = .127) either. Hence, CBT + ERT was not significantly superior to CBT alone regarding therapist-reported improvement.

4. Discussion

4.1. Principal findings

The first aim of this pilot study was to examine the acceptability and feasibility of an internet-based ERT added to CBT for adolescents with depressive and anxiety disorders aged 13–18 and their therapists. Overall, participants were positive about the program and reported moderate satisfaction with the intervention. 76.9 % reported a good quality and 61.5 % stated they received the help they had hoped. However, only 30.8 % would recommend the intervention to friends. Mean adherence (66.5 %) was somewhat lower than the mean adherence rate found in a recent meta-analysis on internet-based CBT for anxiety and depression in adolescents and young people (76.9 %; 10 studies). The number of treatment completers (52.4 %) was largely similar to the mean results in the meta-analysis (57.1 %; 19 studies) (Christ et al., 2020). All therapists stated that the program met all or most of their needs, and 83.3 % would use ERT in the future for their anxious and/or depressed adolescent patients. System usability for therapists was rated as acceptable as well.

Overall, results demonstrated that ERT combined with CBT appears feasible and sufficiently acceptable to deliver in a larger clinical setting. However, although treatment adherence and patients' satisfaction were adequate, both could further be improved. In general, therapist guidance and program content are factors associated with adolescent satisfaction and adherence in digital health interventions (Lehtimaki et al., 2021; Musiat et al., 2021; Smart et al., 2021). Previous studies have underlined the importance of online feedback being personalized, genuine, real, and relevant to users' own experience to improve adherence and motivation in internet-based treatments (Richards et al., 2018; Beatty and Binnion, 2016). Possibly, adherence could be optimized in future research by taking these feedback characteristics into account. Furthermore, it is important to gain insight in specific factors and users' preferences associated with moderate satisfaction found in this study. These factors should be taken into account in future research. This is in line with the ideals of user-centered design, which is a process of e-health design that involves end-users throughout development (Still and Crane, 2017). Inclusion of participants during content design and development would likely be beneficial in creating a more acceptable, feasible and effective intervention.

The second aim was to determine whether an RCT on the effectiveness of an add-on internet-based ERT was feasible regarding recruitment rates, patient acceptability and study adherence rates. Regarding recruitment rates, we found that 76.5 % of eligible respondents participated in the study. From 64.1 % of the included participants one parent participated. At all time-points, >80 % of the participants and parents completed the assessments. Accordingly, loss to follow-up was limited. These findings are above our a priori threshold criteria for feasibility and indicate that participating in the study was acceptable for participants. The study design was considered feasible; therefore, no changes in the study procedures are deemed necessary. However, in a time span of nineteen months only 39 respondents enrolled in the study, which underlines the importance of improving the recruitment strategy. In a future RCT, this could be done by using a multi-center design and by implementing a generic eligibility screening procedure used by all psychologists performing intakes.

The third aim of this study was to provide a first estimate of the effectiveness of CBT + ERT, as compared with CBT alone. In line with our hypothesis, we found a significant reduction of self-reported depressive symptoms and anxiety symptoms in the CBT + ERT group compared to the control (CBT) group, with moderate effect sizes. Moreover, the addition of ERT to CBT was more effective in increasing the use of adaptive ER strategies and decreasing the use of maladaptive ER strategies compared to CBT alone. Hence, our preliminary results indicate that ERT may enhance the effectiveness of CBT in adolescents with depressive or anxiety disorders. These findings correspond with previous meta-analyses in adults and adolescents, indicating that focus on ER is effective in improving ER strategies (Eadeh et al., 2021), and that adaptive ER strategies are related to a decrease in symptoms of anxiety disorders and depression (Schäfer et al., 2016). Regarding parent-reported depressive symptoms, anxiety symptoms, and internalizing problems, no significant results were found.

Surprisingly, the CBT group did not show a significant change in depressive symptoms, anxiety symptoms, and ER strategies at 6-months follow-up. This contradicts extensive research demonstrating CBT to be effective in treating depressive and anxiety disorders in adolescents (e.g., Bennett et al., 2016; Oud et al., 2019). No differences in treatment minutes between both groups were found. Furthermore, a post-hoc chi-square test showed no significant difference in the education level of psychologists between both groups (X2(1) = 1.88, p = .170), although the CBT group showed a higher percentage of highly trained psychologists (50.0 %) than the CBT + ERT group (28.6 %). Highly trained psychologists were defined as psychologists with a postdoctoral degree, or being currently in postdoctoral training.

A possible explanation of the absent treatment effect of CBT may be found in the timing of the data-collection, which took place during the COVID-19 pandemic and several lockdowns. In this period, adolescents were often home-bound, had far less face-to-face social contact, and had to follow their school curriculum online. In addition, the context and frequency of the CBT treatment often changed due to the Covid-19 measures: sessions sometimes changed from face-to-face to screen-to-screen, and, in some lockdowns, sessions were less frequent. This recent COVID-19 period was characterized by an increased global prevalence of depressive and anxiety symptoms in the general adolescent population (Racine et al., 2021). Furthermore, existing mental health problems and depressive symptoms were found to have increased during the pandemic (Barendse et al., 2021; Youngminds, 2020). These results suggest that although CBT did not reduce depressive and anxiety symptoms compared to baseline in the current study, it may have protected adolescents from further deterioration in this stressful period. During the pandemic, the World Health Organization advised adolescents to develop strategies for emotional regulation and to continue appropriate lifestyles (Guessoum et al., 2020; WHO, 2020). Correspondingly, the content of ERT may have fitted the needs of adolescents during the COVID-19 pandemic, and may therefore have led to a decrease in symptoms.

4.2. Strengths and limitations

The present study has several strengths. To our knowledge, it is the first study that examined the feasibility and preliminary effectiveness of an internet-based ERT added to CBT in adolescent patients (13–18) with depression or anxiety disorders. Furthermore, validated patient-reported, parent-reported and clinician-reported outcome measures were used, and loss to follow-up was limited.

This study also has several limitations. First, the sample size was small. Hence, power to establish significant differences was limited, and results regarding effectiveness remain preliminary. Second, there was no active control group added to CBT in the control condition, which makes it impossible to rule out a potential placebo effect of the online ERT. Lastly, this study took place during the COVID-19 pandemic, which may have interfered with the results.

5. Conclusion

The addition of ERT to CBT was found to be feasible and acceptable by patients and therapists. Furthermore, our preliminary results indicate its potential effectiveness in improving anxiety symptoms, depressive symptoms, and the use of ER strategies in adolescent patients with depressive or anxiety disorders. Furthermore, the study design was found to be feasible, although future research should make an effort to improve recruitment rates. To optimize satisfaction and adherence, users' preferences and optimal content design should be identified in focus groups. Furthermore, development of an improved internet-based ERT program in collaboration with adolescents, and its evaluation in a larger, multicenter randomized controlled trial is necessary (Emmelkamp et al., 2022).

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

We wish to thank all patients who participated in this study.

References

  1. Achenbach T.M., Rescorla L.A. University of Vermont, Research Center for Children, Youth, & Families; Burlington, VT: 2001. Manual for the ASEBA School-age Forms and Profiles. [Google Scholar]
  2. Ahmed S.P., Bittencourt-Hewitt A., Sebastian C.L. Neurocognitive based of emotion regulation development in adolescence. Dev. Cogn. Neurosci. 2015;15:11–25. doi: 10.1016/j.dcn.2015.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aldao A., Nolen-Hoeksema S., Schweizer S. Emotion-regulation strategies across psychopathology: a meta-analytic review. Clin. Psychol. Rev. 2010;30(2):217–237. doi: 10.1016/j.cpr.2009.11.004. [DOI] [PubMed] [Google Scholar]
  4. Bae Y. Test review: children’s depression inventory 2 (CDI 2) J. Psychoeduc. Assess. 2012;30(3):304–308. doi: 10.1177/0734282911426407. [DOI] [Google Scholar]
  5. Barendse M., Flannery J., Cavanagh C., Aristizabal M., Becker S.P., Berger E., Pfeifer J. PsyArXiv [Preprint]; 2021. Longitudinal Change in Adolescent Depression and Anxiety Symptoms From Before to During the COVID-19 Pandemic: An International Collaborative of 12 Samples. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Beatty L., Binnion C. A systematic review of predictors of, and reasons for, adherence to online psychological interventions. Int. J. Behav. Med. 2016;23(6):776–794. doi: 10.1007/s12529-016-9556-9. [DOI] [PubMed] [Google Scholar]
  7. Beck A.T. The current state of cognitive therapy: a 40-year retrospective. Arch. Gen. Psychiatry. 2005;62(9):953–959. doi: 10.1001/archpsyc.62.9.953. [DOI] [PubMed] [Google Scholar]
  8. Bennett K., Manassis K., Duda S., Bagnell A., Bernstein G.A., Garland E.J., Wilansky P.… Treating child and adolescent anxiety effectively: overview of systematic reviews. Clin. Psychol. Rev. 2016;50:80–94. doi: 10.1016/j.cpr.2016.09.006. [DOI] [PubMed] [Google Scholar]
  9. Berking M., Ebert D., Cuijpers P., Hofmann S.G. Emotion regulation skills training enhances the efficacy of inpatient cognitive behavioral therapy for major depressive disorder: a randomized controlled trial. Psychother. Psychosom. 2013;82(4):234–245. doi: 10.1159/000348448. [DOI] [PubMed] [Google Scholar]
  10. Berman S.L., Weems C.F., Silverman W.K., Kurtines W.M. Predictors of outcome in exposure-based cognitive and behavioral treatments for phobic and anxiety disorders in children. Behav. Ther. 2000;31(4):713–731. [Google Scholar]
  11. Birmaher B., Khetarpal S., Brent D., Cully M., Balach L., Kaufman J., Neer S.M. The screen for child anxiety related emotional disorders (SCARED): scale construction and psychometric characteristics. J. Am. Acad. Child Adolesc. Psychiatry. 1997;36(4):545–553. doi: 10.1097/00004583-199704000-00018. [DOI] [PubMed] [Google Scholar]
  12. Blakemore S.J. The social brain in adolescence. Nat. Rev. Neurosci. 2008;9(4):267–277. doi: 10.1038/nrn2353. 10.10.38/nrn2353. [DOI] [PubMed] [Google Scholar]
  13. Bodden D., Braet C., Stikkelbroek Y. Hogrefe Uitgevers; Amsterdam: 2016. CDI-2: Screeningsvragenlijst voor depressie bij kinderen en jongeren (Nederlandse bewerking) [Google Scholar]
  14. Braet C., Cracco E., Theuwis L. Hogrefe Uitgevers; Amsterdam: 2013. Vragenlijst over emotieregulatie bij kinderen en jongeren [Self-report Questionnaire to Assess Children’s and Adolescents’ Emotion Regulation Strategies Dutch Version] [Google Scholar]
  15. Buitelaar J.K., Van Keulen M., Nauta M.H., Stikkelbroek Y.A.J., Ruiter M., Van Duin D. Multidisciplinaire richtlijn: Depressie bij Jeugd Addendum. 2009. https://assets-,sites.trimbos.nl/docs/1e4da111-1203-4bab-bba4-2a6897f163ee.pdf
  16. Busner J., Targum S.D., Miller D.S. The clinical global impressions scale: errors in understanding and use. Compr. Psychiatry. 2009;50(3):257–262. doi: 10.1016/j.comppsych.2008.08.005. [DOI] [PubMed] [Google Scholar]
  17. Casey B.J., Getz S., Galvan A. The adolescent brain. Dev. Rev. 2008;28(1):62–77. doi: 10.1016/j.dr.2007.08.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Central Bureau for Statistics Internet jongeren; toegang, gebruik en faciliteiten, 2012-2019. 2019. https://jmopendata.cbs.nl/#/JM/nl/dataset/20255NED/table?ts=1627374110866
  19. Christ C., Schouten M.J., Blankers M., van Schaik D.J., Beekman A.T., Wisman M.A., Dekker J.J.M.… Internet and computer-based cognitive behavioral therapy for anxiety and depression in adolescents and young adults: systematic review and meta-analysis. J. Med. Internet Res. 2020;22(9) doi: 10.2196/17831. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Cohen J. 2nd edn. Taylor & Francis; Chicago: 1988. Statistical Power Analysis for the Behavioral Sciences. [Google Scholar]
  21. Compas B.E., Jaser S.S., Bettis A.H., Watson K.H., Gruhn M.A., Dunbar J.P., Thigpen J.C.… Coping, emotion regulation, and psychopathology in childhood and adolescence: a meta-analysis and narrative review. Psychol. Bull. 2017;143(9):939. doi: 10.1037/bu10000110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Costello E.J., Mustillo S., Erkanli A., Keeler G., Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Arch. Gen. Psychiatry. 2003;60(8):837–844. doi: 10.1001/archpsyc.60.8.837. [DOI] [PubMed] [Google Scholar]
  23. Cracco E., Van Durme K., Braet C. Validation of the FEEL-KJ: an instrument to measure emotion regulation strategies in children and adolescents. Plos One. 2015;10:1–18. doi: 10.1371/journal.pone.0137080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Cracco E., Goossens L., Braet C. Emotion regulation across childhood and adolescence: evidence for a maladaptive shift in adolescence. Eur. Child Adolesc. Psychiatry. 2017;26:909–921. doi: 10.1007/s00787-017-0952-8. [DOI] [PubMed] [Google Scholar]
  25. Craighead W.E., Smucker M.R., Craighead L.W., Ilardi S.S. Factor analysis of the children's depression inventory in a community sample. Psychol. Assess. 1998;10(2):156. [Google Scholar]
  26. Crowe K., McKay D. Efficacy of cognitive-behavioral therapy for childhood anxiety and depression. J. Anxiety Disord. 2017;49:76–87. doi: 10.1016/j.janxdis.2017.04.001. [DOI] [PubMed] [Google Scholar]
  27. De Beurs E., Carlier I.V., van Hemert A.M. Approaches to denote treatment outcome: clinical significance and clinical global impression compared. Int. J. Methods Psychiatr. Res. 2019;28(4) doi: 10.1002/mpr.1797. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. De Brey H. A cross-national validation of the client satisfaction questionnaire: the dutch experience. Eval. Progr. Plan. 1983;6(3–4):395–400. doi: 10.1016/0149-7189(83)90018-6. [DOI] [PubMed] [Google Scholar]
  29. Eadeh H.M., Breaux R., Nikolas M.A. A meta-analytic review of emotion regulation focused psychosocial interventions for adolescents. Clin. Child. Fam. Psychol. Rev. 2021;24(4):684–706. doi: 10.1007/s10567-021-00362-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Ebert D.D., Zarski A.C., Christensen H., Stikkelbroek Y.A.J., Cuijpers P., Berking M., Riper H. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PloS One. 2015;10(3) doi: 10.1371/journal.pone.0119895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Ehrenreich-May J., Rosenfield D., Queen A.H., Kennedy S.M., Remmes C.S., Barlow D.H. An initial waitlist-controlled trial of the unified protocol for the treatment of emotional disorders in adolescents. J. Anxiety Disord. 2017;46:46–55. doi: 10.1016/j.janxdis.2016.10.006. [DOI] [PubMed] [Google Scholar]
  32. Emmelkamp J., Wisman M.A., Beuk N.J.M., Stikkelbroek Y.A.J., Nauta M.H., Dekker J.J.M., Christ C. Effectiveness of an add-on guided internet-based emotion regulation training (E-TRAIN) in adolescents with depressive and/or anxiety disorders: study protocol for a multicenter randomized controlled trial. BMC Psychiatry. 2022;22(1):1–13. doi: 10.1186/s12888-022-04291-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Essau C.A., Conradt J., Petermann F. Frequency, comorbidity, and psychosocial impairment of anxiety disorders in German adolescents. J. Anxiety Disord. 2000;14(3):263–279. doi: 10.1016/s0887-6185(99)00039-0. [DOI] [PubMed] [Google Scholar]
  34. Essau C.A., Lewinsohn P.M., Olaya B., Seeley J.R. Anxiety disorders in adolescents and psychosocial outcomes at age 30. J. Affect. Disord. 2014;163:125–132. doi: 10.1016/j.jad.2013.12.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Forkmann T., Scherer A., Boecker M., Pawelzik M., Jostes R., Gauggel S. The clinical global impression scale and the influence of patient or staff perspective on outcome. BMC Psychiatry. 2011;11(1):1–7. doi: 10.1186/1471-244X-11-83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Garber J., Weersing V.R. Comorbidity of anxiety and depression in youth: implications for treatment and prevention. Clin. Psychol. Sci. Pract. 2010;17(4):293. doi: 10.1111/j.1468-2850.2010.01221.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Grist R., Croker A., Denne M., Stallard P. Technology delivered interventions for depression and anxiety in children and adolescents: a systematic review and meta-analysis. Clin. Child. Fam. Psychol. Rev. 2019;22(2):147–171. doi: 10.1007/s10567-018-0271-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Grob A., Smolenski C. Huber Verlag; Bern: 2005. Fragebogen zur erhebung der emotionsregulation bei kindern und jugendlichen (FEEL-KJ) [Google Scholar]
  39. Gross J.J., Thompson R.A. In: Handbook of Emotion Regulation. Gross J.J., editor. Guilford Press; New York: 2007. Emotion regulation: conceptual foundations; pp. 3–24. [Google Scholar]
  40. Guessoum S.B., Lachal J., Radjack R., Carretier E., Minassian S., Benoit L., Moro M.R. Adolescent psychiatric disorders during the COVID-19 pandemic and lockdown. Psychiatry Res. 2020 doi: 10.1016/j.psychres.2020.113264. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Guy W. DHEW publication ADM 76–338. Government Printing Office; Washington, DC: 1976. “Clinical Global Impressions” ECDEU Assessment Manual for Psychopharmacology, Revised. [Google Scholar]
  42. Hale W.W., Crocetti E., Raaijmakers Q.A., Meeus W.H. A meta-analysis of the cross- cultural psychometric properties of the screen for child anxiety related emotional disorders (SCARED) J. Child Psychol. Psychiatry. 2011;52(1):80–90. doi: 10.1111/j.1469-7610.2010.02285.x. [DOI] [PubMed] [Google Scholar]
  43. Harris P.L., De Rosnay M., Pons F. In: Handbook of Emotions 4th. Lewis M., Havilang-Jones F., Feldman Barrett M., editors. Guilford Press; New York: 2016. Understanding emotion; pp. 293–306. [Google Scholar]
  44. Horn A.B., Pössel P., Hautzinger M. Promoting adaptive emotion regulation and coping in adolescence: a school-based programme. J. Health Psychol. 2011;16(2):258–273. doi: 10.1177/1359105310372814. [DOI] [PubMed] [Google Scholar]
  45. James A.C., Reardon T., Soler A., James G., Creswell C. Cognitive behavioural therapy for anxiety disorders in children and adolescents. Cochrane Database Syst. Rev. 2020;11 doi: 10.1002/14651858.CD013162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Jaycox L.H., Stein B.D., Paddock S., Miles J.N., Chandra A., Meredith L.S., Burnam M.A.… Impact of teen depression on academic, social, and physical functioning. Pediatrics. 2009;124:596–605. doi: 10.1542/peds.2008-3348. [DOI] [PubMed] [Google Scholar]
  47. Johnson D., Dupuis G., Piche J., Clayborne Z., Colman I. Adult mental health outcomes of adolescent depression: a systematic review. Depress. Anxiety. 2018;35(8):700–716. doi: 10.1002/da.22777. [DOI] [PubMed] [Google Scholar]
  48. Kendall P.C., Peterman J.S. CBT for adolescents with anxiety: mature yet still developing. Am. J. Psychiatry. 2015;172(6):519–530. doi: 10.1176/appi.ajp.2015.14081061. [DOI] [PubMed] [Google Scholar]
  49. Klein J.B., Jacobs R.H., Reinecke M.A. Cognitive-behavioral therapy for adolescent depression: a meta-analytic investigation of changes in effect-size estimates. J. Am. Acad. Child Adolesc. Psychiatry. 2007;46:1403–1413. doi: 10.1097/chi.0b013e3180592aaa. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Kovacs M. Multi-Health Systems; New York: 1992. Children’s Depression Inventory (CDI) Manual. [Google Scholar]
  51. Lehtimaki S., Martic J., Wahl B., Foster K.T., Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment. Health. 2021;8(4) doi: 10.2196/25847. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Lindqvist K., Mechler J., Carlbring P., Lilliengren P., Falkenström F., Andersson G., Philips B.… Affect-focused psychodynamic internet-based therapy for adolescent depression: randomized controlled trial. J. Med. Internet Res. 2020;22(3) doi: 10.2196/18047. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Linehan M.M. Guilford; New York: 1993. Cognitive-behavioral Treatment of Borderline Personality Disorder. [Google Scholar]
  54. Merikangas K.R., Nakamura E.F., Kessler R.C. Epidemiology of mental disorders in children and adolescents. Dialogues Clin. Neurosci. 2009;11(1):7. doi: 10.31887/DCNS.2009.11.1/krmerikangas. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Muris P., Bodden D., Hale W., Birmaher B., Mayer B. Boom test uitgevers; Amsterdam, Nederland: 2007. SCARED-NL. Vragenlijst over angst en bang-zijn bij kinderen en adolescenten. Handleiding bij de gereviseerde Nederlandse versie van de Screen for Child Anxiety Related Emotional Disorders. [Google Scholar]
  56. Musiat P., Johnson C., Atkinson M., Wilksch S., Wade T. Impact of guidance on intervention adherence in computerised interventions for mental health problems: a meta-analysis. Psychol. Med. 2021;1–12 doi: 10.1017/S0033291721004621. [DOI] [PubMed] [Google Scholar]
  57. Oud M., De Winter L., Vermeulen-Smit E., Bodden D., Nauta M., Stone L., Stikkelbroek Y.… Effectiveness of CBT for children and adolescents with depression: a systematic review and meta-regression analysis. Eur. Psychiatry. 2019;57:33–45. doi: 10.1016/j.eurpsy.2018.12.008. [DOI] [PubMed] [Google Scholar]
  58. Pasarelu C.R., Andersson G., Nordgren L.B., Dobrean A. Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials. Cogn. Behav. Ther. 2007;46:1–28. doi: 10.1080/16506073.2016.1231219. [DOI] [PubMed] [Google Scholar]
  59. Pasarelu C.R., Dobrean A., Andersson G., Zaharie G.C. Feasibility and clinical utility of a transdiagnostic internet-delivered rational emotive and behavioral intervention for adolescents with anxiety and depressive disorders. Internet Interv. 2021;26 doi: 10.1016/j.invent.2021.100479. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Polanczyk G.V., Salum G.A., Sugaya L.S., Caye A., Rohde L.A. Annual research review: a meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. J. Child Psychol. Psychiatry Allied Discip. 2015;56:345–365. doi: 10.1111/jcpp.12381. [DOI] [PubMed] [Google Scholar]
  61. Racine N., McArthur B.A., Cooke J.E., Eirich R., Zhu J., Madigan S. Global prevalence of depressive and anxiety symptoms in children and adolescents during COVID-19: a meta-analysis. JAMA Pediatr. 2021;175(11):1142–1150. doi: 10.1001/jamapediatrics.2021.2482. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Richards D., Dowling M., O'Brien E., Vigano N., Timulak L. Significant events in an internet-delivered (Space from Depression) intervention for depression. Couns. Psychother. Res. 2018;18(1):35–48. [Google Scholar]
  63. Sakiris N., Berle D. Systematic review and meta-analysis of the unified protocol as a transdiagnostic emotion regulation based intervention. Clin. Psychol. Rev. 2019;72 doi: 10.1016/j.cpr.2019.101751. [DOI] [PubMed] [Google Scholar]
  64. Sandín B., García-Escalera J., Valiente R.M., Espinosa V., Chorot P. Clinical utility of an internet-delivered version of the unified protocol for transdiagnostic treatment of emotional disorders in adolescents (iUP-A): a pilot open trial. Int. J. Environ. Res. Public Health. 2020;17:8306. doi: 10.3390/ijerph17228306. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Sauro J., Lewis J.R. Quantifying the User Experience. Elsevier; Amsterdam, The Netherlands: 2016. Quantifying user research; pp. 9–18. [Google Scholar]
  66. Schäfer J.O., Naumann E., Holmes E.A., Tuschen-Caffier B., Samson A.C. Emotion regulation strategies in depressive and anxiety symptoms in youth: a meta-analytic review. J. Youth Adolesc. 2016;46:261–276. doi: 10.1007/s10964-016-0585-0. [DOI] [PubMed] [Google Scholar]
  67. Seager I., Rowley A.M., Ehrenreich-May J. Targeting common factors across anxiety and depression using the unified protocol for the treatment of emotional disorders in adolescents. J. Ration. Emot. Cogn. Behav. Ther. 2014;32:67–83. doi: 10.1007/s10942-014-0185-5. [DOI] [Google Scholar]
  68. Seligman L.D., Ollendick T.H. Cognitive-behavioral therapy for anxiety disorders in youth. Child Adolesc. Psychiatr. Clin. N. Am. 2011;20(2):217–238. doi: 10.1016/j.chc.2011.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Sherman J.A., Ehrenreich-May J. Changes in risk factors during the unified protocol for transdiagnostic treatment of emotional disorders in adolescents. Behav. Ther. 2020;51:869–881. doi: 10.1016/j.beth.2019.12.002. [DOI] [PubMed] [Google Scholar]
  70. Sloan E., Hall K., Moulding R., Bryce S., Mildred H., Staiger P.K. Emotion regulation as a transdiagnostic treatment construct across anxiety, depression, substance, eating and borderline personality disorders: a systematic review. Clin. Psychol. Rev. 2017;57:141–163. doi: 10.1016/j.cpr.2017.09.002. [DOI] [PubMed] [Google Scholar]
  71. Smart K., Smith L., Harvey K., Waite P. The acceptability of a therapist-assisted internet-delivered cognitive behaviour therapy program for the treatment of anxiety disorders in adolescents: a qualitative study. Eur. Child Adolesc. Psychiatry. 2021;1–13 doi: 10.1007/s00787-021-01903-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Southam-Gerow M.A. The Guilford Press; New York: 2013. Emotion Regulation in Children and Adolescents: A Practitioner’s Guide. [Google Scholar]
  73. Still B., Crane K. Taylor & Francis Group; Goca Raton, FL: 2017. Fundamentals of User-Centered Design: A Practical Approach. [Google Scholar]
  74. Suveg C., Kendall P.C., Comer J.S., Robin J. Emotion-focused cognitive-behavioral therapy for anxious youth: a multiple-baseline evaluation. J. Contemp. Psychother. 2006;36:77–85. [Google Scholar]
  75. Suveg C., Sood E., Comer J.S., Kendall P.C. Changes in emotion regulation following cognitive-behavioral therapy for anxious youth. J. Clin. Child Adolesc. Psychol. 2009;38(3):390–401. doi: 10.1080/15374410902851721. [DOI] [PubMed] [Google Scholar]
  76. Thompson R.A. Emotion regulation: a theme in search of definition. Monogr. Soc. Res. Child Dev. 1994;59:25–52. doi: 10.1111/j.1540-5834.1994.tb01276.x. [DOI] [PubMed] [Google Scholar]
  77. Twisk J.W. Cambridge University Press; 2013. Applied Longitudinal Data Analysis for Epidemiology: A Practical Guide. [Google Scholar]
  78. Verhulst F.C., Van der Ende J. ASEBA Nederland; Rotterdam: 2013. Handleiding ASEBA-Vragenlijsten voor leeftijden 6 t/m 18 jaar: CBCL/6-18, YSR en TRF. [Google Scholar]
  79. Watanabe N., Hunot V., Omori I.M., Churchill R., Furukawa T.A. Psychotherapy for depression among children and adolescents: a systematic review. Acta Psychiatr. Scand. 2007;116(2):84–95. doi: 10.1111/j.1600-0447.2007.01018.x. [DOI] [PubMed] [Google Scholar]
  80. Wilcox R. Trimming and Winsorization. Encyclopedia of Biostatistics. 2005. p. 8. [Google Scholar]
  81. World Health Organization . Who.int; 2020. Coping with stress during the 2019-nCoV outbreak. [Google Scholar]
  82. World Health Organization . Who.int; 2021. Adolescent mental health.https://www.who.int/news-room/fact-sheets/detail/adolescent-mental-health [Google Scholar]
  83. Young K.S., Sandman C.F., Craske M.G. Positive and negative emotion regulation in adolescence: links to anxiety and depression. Brain Sci. 2019;9(4):76. doi: 10.3390/brainsci9040076. [DOI] [PMC free article] [PubMed] [Google Scholar]
  84. Youngminds . Youngminds.org.uk; 2020. Coronavirus: impact on young people with mental health needs.https://youngminds.org.uk/media/3708/coronavirus-report_march2020.pdf March. [Google Scholar]
  85. Zaider T.I., Heimberg R.G., Fresco D.M., Schneier F.R., Liebowitz M.R. Evaluation of the clinical global impression scale among individuals with social anxiety disorder. Psychol. Med. 2003;33(4):611–622. doi: 10.1017/S0033291703007414. [DOI] [PubMed] [Google Scholar]
  86. Zeman J., Cassano M., Perry-Parrish C., Stegall S. Emotion regulation in children and adolescents. J. Dev. Behav. Pediatr. 2006;27(2):155–168. doi: 10.1097/00004703-200604000-00014. 0196-206X/06/2702-0155. [DOI] [PubMed] [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES