Skip to main content
NCBI home page
Internet Interventions logoLink to Internet Interventions
. 2018 Jan 6;11:30–40. doi: 10.1016/j.invent.2018.01.001

Internet-based cognitive behavioral therapy for adolescents with anxiety disorders: A feasibility study

Silke Stjerneklar a,⁎, Esben Hougaard a, Amalie D Nielsen b, Majken M Gaardsvig a, Mikael Thastum a
PMCID: PMC6084871  PMID: 30135757

Abstract

Background

Cognitive behavioral therapy (CBT) is a well-documented effective method for the treatment of anxiety disorders in children and adolescents. While internet based CBT (ICBT) programs for adults have been widely investigated, research on ICBT programs for anxiety disorders in youth is still in an early phase: To date, no such program has been developed or evaluated in Denmark.

Aim

As preparation for a randomized controlled efficacy trial, this study aimed at evaluating the feasibility of a translated and adapted version of the ICBT program ‘Chilled Out’ for adolescents with anxiety disorders, developed at Macquarie University, Australia.

Methods

At the University Clinic in Aarhus, Denmark, we conducted a feasibility study with six adolescents with a primary anxiety disorder. The 12-week ICBT intervention consisted of eight online modules. Participants received weekly phone calls from a clinical psychologist. Semi-structured interviews on participant's experiences of the program were administered post-treatment and at three-month follow-up. Outcome was evaluated post-treatment and at follow-up using diagnostic interview and questionnaires.

Results

Five of the six participants completed the program. Participants were generally satisfied with the program and the majority would recommend it to others. Preliminary clinical outcome results indicated moderate to large improvements (e.g., a standardized mean difference from pre- to post-treatment of d = 1.54 on the clinical severity rating of their primary anxiety disorder).

Conclusion

Results from this study indicate that a translated and revised version of the Chilled Out program could be a feasible psychological intervention for Danish adolescents with anxiety disorders.

Keywords: Anxiety disorders, Internet-based, Cognitive behavioral therapy, Feasibility, Adolescents

Highlights

  • •

    Participants demonstrated large symptom reductions in primary anxiety diagnosis.

  • •

    Two participants no longer met criteria for their primary anxiety diagnosis (33%).

  • •

    Three participants were free of their secondary anxiety diagnoses (50%).

  • •

    Adolescents and their parents generally expressed satisfaction with the intervention.

  • •

    The majority of participants and parents would recommend the intervention to others.

1. Introduction

Anxiety disorders are among the most common psychiatric disorders in adolescence with 5–12% of youths from western countries suffering from an anxiety disorder (Beesdo et al., 2009; Costello et al., 2011). Besides being distressful in itself, anxiety disorders in youths may lead to problems in relation to school activities, peer relations and family function (Davis et al., 2008). If left untreated, anxiety often has a chronic developmental course, predicting anxiety and depression in adulthood (Kessler et al., 2005; Muris, 2006; Woodward and Fergusson, 2001).

Cognitive behavioral therapy (CBT) is the best-documented treatment for anxiety disorders with studies on youth showing large treatment effects (Cartwright-Hatton et al., 2004; James et al., 2013). However, studies indicate that only a small percentage of anxious youths receives treatment, despite the obvious need for health care services for this population (Essau et al., 2000; Merikangas et al., 2010; Wang et al., 2007). A major reason could be the lack of treatment facilities. But adolescents may also be especially reluctant to seek treatment, even if treatment facilities are available (Tillett, 2005). Illustrating this, a cross-sectional school-based study of >11.000 Norwegian adolescents aged 15 to 16 years found that only 34% of those with high levels of mental health symptoms reported help-seeking in the previous year (Zachrisson et al., 2006).

Among the most frequently mentioned barriers for adolescents to seek treatment are perceived social stigma and fear of peer rejection (Collins et al., 2004; Gulliver et al., 2010; Rickwood et al., 2005; Booth et al., 2004; Jaycox et al., 2006), preference for self-reliance (Rickwood et al., 2007; Rickwood and Braithwaite, 1994; Wilson et al., 2011; Booth et al., 2004), confidentiality and privacy concerns (Gulliver et al., 2010; Booth et al., 2004), worries concerning treatment costs, transportation or waiting times (Elliott and Larson, 2004), and lacking accessibility of available mental health services (Gulliver et al., 2010, Booth et al., 2004). Consequently, it is of foremost importance that adolescents are provided with interventions that strive to overcome the above-mentioned help-seeking barriers to facilitate treatment accessibility and participation young people might believe professional mental health care is not helpful.

Within the last decade, there has been a substantial increase in the development and use of internet based CBT (ICBT) programs, as a mean to reduce costs and enhance accessibility of psychological interventions. ICBT interventions most often provide comparable therapeutic content to that which is delivered in regular face-to-face CBT, modified for the online delivery and usually presented in modules on a weekly basis (Andersson and Titov, 2014; Barak et al., 2009; Ritterband et al., 2003). Typically, clients work through a highly-structured program learning cognitive behavioral techniques, strategies and activities to gradually manage their anxiety. ICBT interventions can be either self-directed or therapist-guided, with the term ‘guidance’ referring to a continuum ranging from automated reminders with no human interaction to programs in which a clinician supports the client during the course of the intervention, either through time delayed email correspondence, brief scheduled phone calls or real-time chat (Andersson, 2016). The therapeutic support commonly appears as encouragement, problem solving and/or feedback on ‘offline’ assignments (Andersson, 2014; Holländare et al., 2016).

The delivery of therapy online has many advantages for clients and clinicians including (1) increased accessibility, (2) greater anonymity, (3) flexibility, (4) reduced expenses, (5) eliminated travel time, and (6) interactivity (Christensen et al., 2014; Cunningham et al., 2009). As such, ICBT has the potential to overcome many of the above-mentioned treatment barriers faced by the adolescent population. Moreover, as ‘digital natives’ most adolescents are highly familiar with the use of digital devices (computers, tablets, smartphones), engaging in daily activities on the internet and naturally seeking out health information online (Ho et al., 2016; Stallard et al., 2010; Subrahmanyam and Lin, 2007). The inclusion of ICBT as a treatment option for adolescents with anxiety disorders seems thus to pose a promising solution to the small number of adolescents receiving treatment today.

A growing body of research now demonstrates substantial evidence for the efficacy of internet-based cognitive behavioral therapy (ICBT) interventions for several anxiety disorders in adults (Hedman et al., 2012; Andersson, 2016; Richards et al., 2015; Spek et al., 2007; Christensen et al., 2014). Although ICBT for anxiety in adolescents is still in an early phase, emerging research show promising results. Several reviews (Reyes-Portillo et al., 2014; Richardson et al., 2010; Stasiak et al., 2016) and meta-analyses of between 7 and 27 studies (N = 404–3389) of children and adolescents with anxiety and/or depressive symptoms (Ebert et al., 2015; Pennant et al., 2015; Podina et al., 2016; Ye et al., 2014) now support the effectiveness of ICBT with anxious adolescents showing moderate-to-large effects on anxiety symptoms compared to no treatment (g = 0.53–1.41) across different analyses (Hollis et al., 2017).

When looking specifically at ICBT with adolescents, studies are sparse (Tillfors et al., 2011; Spence et al., 2008; Spence et al., 2011; Lenhard et al., 2014; Lenhard et al., 2017). Most recently, Lenhard et al. (2017) reported results from a single-blinded randomized controlled trial with sixty-seven adolescents (aged 12–17) suffering from obsessive-compulsive disorder (OCD). In this trial, a 12-week clinician and parent supported ICBT-intervention proved superior to wait-list controls on the semi-structured clinician-administered interview Children's Yale-Brown Obsessive-Compulsive Scale (Scahill et al., 1997) with an effect-size of d = 0.69 (p < 0.001).

The present study succeeded a project phase in which the ICBT program under investigation, the Chilled Out program, was translated and revised according to Danish language. Revisions consisted primarily of the recording of new video material with Danish adolescents and professionals; all other translations were done as close to the Australian program as possible. Chilled Out was recently developed at Macquarie University, Sydney, Australia, from the Cool Teens CD-ROM (Wuthrich et al., 2013; Cunningham, 2008), previously demonstrating good efficacy in a randomized controlled trial with 43 adolescents and their mothers (Wuthrich et al., 2012). To the best of our knowledge, the Chilled Out program has not been evaluated earlier and as such, this is the first study of the program.

1.1. Aim

As preparation for an upcoming randomized controlled efficacy trial, the main objective of this study was to examine the feasibility, i.e. attrition rate, program activity, user experience, therapeutic support, treatment satisfaction, etc. and preliminary clinical outcome of a Danish version of the ICBT program ‘Chilled Out’ for adolescents with anxiety disorders.

2. Methods

2.1. Participants

The study took place within the context of the University Clinic, a training and research facility at the Department of Psychology and Behavioral Sciences, Aarhus University, Denmark. Six adolescents aged 13–17 years with a primary anxiety diagnosis were included. As this was the first study of the Chilled Out program, the degree to which the adolescents would benefit from the presented intervention was unknown. A sample size of six was chosen as it would allow us to test the program feasibility while simultaneously inconveniencing only a small group of participants, should the program show limited effect. Participants were recruited through announcement at the Clinic's website and screened through a brief e-mail sent in by the families describing the adolescent's major problems. If eligible, families were contacted for a structured diagnostic telephone interview. Inclusion criteria were: age between 13 and 17 years; an anxiety disorder as primary diagnosis according to the DSM-IV (Association, 1994); direct access to a home computer with internet; and the ability to read and write in Danish. Exclusion criteria were severe comorbid depression; school absence above 50% for a period of 3 month; drug or alcohol dependence; current self-harm or suicidal ideation; psychotic symptoms; intellectual disability; pervasive developmental disorder; severe conduct disorders; and PTSD as primary disorder. Participants were asked to refrain from other psychotherapeutic treatment during the trial, and if on anxiety medications not to make changes in dose.

2.2. Procedure

Families that met inclusion criteria were contacted and informed of the study processes. After having received both oral and written study information, families signed a written consent form and were included in the study. Designed with varying baseline comparison, the intervention succeeded a baseline waiting period of between 2 and 4 months (mean baseline time was 13.5 weeks; SD = 2.7). Due to a limited timeframe, recruitment was initiated before the program had been completely translated. The differing baseline periods were thus the result of a continuous recruitment process and one collective intervention start date.

Adolescents and parents were assessed with a battery of online self-report questionnaires at time of inclusion (pre-I), before the intervention (pre-II), after the intervention (post), and at three-month follow-up. Additionally, adolescents were asked to fill out The Working Alliance Inventory-Short Form (WAI-S) (Tracey and Kokotovic, 1989) half-way through the intervention. Questionnaires were collected via email using an online data collection platform (SurveyXact). Diagnostic status was assessed at pre-I, pre-II and post with the Anxiety Disorders Interview Schedule for DSM-IV, Child and Parent Versions (ADIS-IV C/P) (Silverman and Albano, 1996) administered over the telephone. For this study, the ADIS-IV interviews were conducted by a clinical psychologist, experienced in using the ADIS-IV, or a trained and supervised graduate psychology students. Additionally, adolescents and parents received separate phone calls at post and follow-up examining program feasibility measures such as user experiences, and treatment satisfaction using a semi-structured interview guide designed for the study. The interviews were performed by master degree psychology students working at the Clinic. Interviews were recorded using Crystal Gears® Ver. 2.00 RTM and transcribed.

Two weeks prior to intervention start, participants were introduced to their assigned therapist during a 20-minute phone call that also summarized the intervention process and shortly introduced the adolescent to the program website and functions. The adolescent then received a letter containing information regarding treatment start date, program information (the program URL, username, and temporary password), the therapist mail address and an overview of time and dates for the future therapist phone calls. All therapist calls were recorded using Crystal Gears® Ver. 2.00 RTM to enable subsequent content coding and analysis. The trial was approved by the Danish Data Protection Agency and by the local Ethics Committee of Central Denmark Region.

2.3. Treatment

The Chilled Out program comprises eight online modules of approximately 30 min each, teaching adolescents to gradually manage their anxiety predominantly through psychoeducation, cognitive restructuring and graded exposure. The program is interactive using a combination of different multimedia channels, i.e. text, audio, illustrations, worksheets, cartoons, and video vignettes, to deliver CBT-inspired activities and exercises such as goal setting, realistic thinking, problem solving, assertiveness, etc. An overview of the module content and suggested homework is presented in Table 1. With the main purpose of normalizing anxiety related feelings and experiences, participants are initially introduced to six video cases of adolescents presenting with different anxiety issues. Throughout the program, these adolescent characters reappear as they are shown applying various program skills and coping strategies to manage their problems. Each module contains a selection of recommended homework practice tasks. Adolescents are encouraged to spend as much time as possible each week completing tasks away from the computer. Once a week, participants are asked to rate 6 statements on a 5-point Likert scale concerning the interference of anxiety in their daily lives. If completed, this short questionnaire enables participants to view a progress chart of their total anxiety score and hence track their change over time. Participants were encouraged to complete the eight modules within a 12-week treatment period, after which the participants would have access to the web site for another 3 months.

Table 1.

Module content and homework practice tasks.

Module title Module content Homework practice tasks
1. Understanding anxiety
  • •

    How to use the program

  • •

    Psychoeducation about anxiety
  • •

    Complete an anxiety self-assessment questionnaire to get overview of present anxiety issues
2. Setting goals
  • •

    Learning to set goals

  • •

    Rewards

  • •

    Measuring feelings on a worry scale
  • •

    Identify goals

  • •

    Make a list of possible rewards
3. Realistic thinking I
  • •

    Linking thoughts and feelings

  • •

    Negative thinking

  • •

    Identifying and challenging unrealistic thoughts (cognitive restructuring)
  • •

    Practice realistic thinking

  • •

    Rewards
4. Stepladders I
  • •

    Identifying and categorizing fears and worries

  • •

    Graded exposure using stepladders
  • •

    Create the first stepladder

  • •

    Plan the execution of the stepladder

  • •

    Practice steps until goal is reached
5. Stepladders II
  • •

    Revising ‘old’ and creating new stepladders

  • •

    Behavioral experiments

  • •

    Help solving stepladder barriers and difficulties
  • •

    Create more stepladders

  • •

    Practice behavioral experiments
6. Realistic thinking II
  • •

    Simplifying realistic thinking (in my mind)

  • •

    Acting as if

  • •

    Surfing emotions
  • •

    Make list of useful questions and thoughts for realistic thinking

  • •

    Continue working on stepladders
7. Other coping skills
  • •

    Problem solving

  • •

    Constructive feedback

  • •

    Assertive communication

  • •

    Calming activities
  • •

    Practice problem solving and assertiveness
8. Staying chilled
  • •

    Skills overview and maintenance

  • •

    Relapse prevention
  • •

    Continue to practice skills
Open in a new tab

Once every week the adolescents received a phone call from a therapist. The primary focus of these weekly phone conversations was on ensuring the adolescents' understanding of the program elements, offering technical assistance, and encouraging participants to continue working with the program during times of low motivation. The therapist had access to participants' program responses and worksheets to guide the feedback and provide corrective information if necessary. Three months after treatment completion the adolescents received a booster phone call from the therapist addressing current use of the program, anxiety management skills, challenges that needed problem solving and the adolescent's thoughts about the future. Primarily based on the adolescent's expressed needs and wishes, during this phone call it was also decided whether the adolescent needed more time to work with the program, i.e. a three-month prolonged access to the website. All phone calls were recorded. The main themes of the conversations and the call duration were noted in the participant files.

Shortly before treatment start parents received The Chilled Out Mentor Companion (Wuthrich, 2014) by regular mail; a booklet describing their role in treatment and providing them with basic knowledge and core strategies of the program (psychoeducation, goal setting, cognitive restructuring, and graded exposure). Parents were encouraged to read the companion parallel to their adolescent working through the program for them to be able to help the adolescent during the intervention. However, it was completely up to each adolescent to decide when, how, or if they wanted their support. Parents were invited to contact the therapist via e-mail in case they had questions regarding the intervention. If needed, parents were offered a short phone call from the therapist.

2.4. Measures

2.4.1. The Anxiety Disorders Interview Schedule for DSM-IV, Child and Parent Versions

The Anxiety Disorders Interview Schedule for DSM-IV, Child and Parent Versions (ADIS-IV C/P) (Silverman and Albano, 1996) is a semi-structured interview used to assess child and adolescent anxiety disorders according to DSM-IV. In addition to the assessment of anxiety disorders, the ADIS-IV makes assessment of other disorders often associated with anxiety possible, including depression, dysthymia, ADHD, oppositional defiant disorder, and conduct disorder. Clinical Severity Ratings (CSR) of the disorders are evaluated by the clinician on a 9-point Likert scale (0 = not at all disturbing; 8 = severely disturbing) considering the evaluations by youth and parents. A CSR of 4 (moderately disturbing) or above is necessary for a diagnosis. The most impairing diagnosis is considered the primary diagnosis. The ADIS-IV has previously demonstrated good to excellent test-retest reliability for the presence of specific diagnoses and for the CSR (Silverman et al., 2001). Administering the interview over the telephone has been found to have high validity and inter-rater reliability (Lyneham and Rapee, 2005).

2.4.2. The Spence Children's Anxiety Scale

The Spence Children's Anxiety Scale (SCAS) (Spence, 1998) is a measure of the adolescent's anxiety symptoms with 38 items, each rated on a 4-point Likert scale from 0 to 3. The scale is filled out both by the adolescent (SCAS-C) and by the each of the parents (SCAS-P). The scale comprises six subscales relating to social phobia (SoP), generalized anxiety disorder (GAD), specific phobia (SP [fear of physical injury]), separation anxiety disorder (SAD), OCD, and panic disorder (PD) together with agoraphobia (AP). The Danish version of SCAS has demonstrated sound psychometric properties in a Danish community and clinical sample (Arendt et al., 2014).

2.4.3. The Child Anxiety Life Inference Scale

The Child Anxiety Life Inference Scale (CALIS) (Lyneham et al., 2013) measures the impact of youths' anxiety on their life and on the family as evaluated separately by the adolescent (nine items) and their parents (16 items) on 5-point Likert scales (0 = not at all; 4 = a great deal). The scale has previously demonstrated satisfactory internal consistency and moderate retest stability (Lyneham et al., 2013).

2.4.4. The Working Alliance Inventory-Short Form

The Working Alliance Inventory-Short Form (WAI-S) (Tracey and Kokotovic, 1989) is a shortened 12-item version of the full 36-item WAI (Horvath and Greenberg, 1989) measuring the therapeutic alliance with three sub-scales, goal (agreement on the goals of therapy), task (agreement on how goals are achieved), and bond (personal attraction between therapist and client). Items are rated on 7-point Likert scales (1 = never; 7 = always). The scale was filled out by the adolescent. For this article, only the general working alliance score is presented, calculated as the average of all item responses with higher scores indicating a stronger alliance. The WAI and WAI-S has previously demonstrated reliability as well as convergent, concurrent, discriminant, and predictive validity (Horvath and Greenberg, 1989; Tracey and Kokotovic, 1989). More recently, the WAI has demonstrated acceptable psychometric properties in the context of child therapy (Kazdin et al., 2005; Kazdin et al., 2006).

2.4.5. Experience of service questionnaire

Post treatment, adolescents and parents were presented with a short questionnaire adapted from The Experience of Service Questionnaire (Brown et al., 2014; Attride-Stirling, 2002) measuring their satisfaction with the intervention. Separate versions were used for adolescents (seven items) and parents (9 items), both rating affirmative statements as ‘not true’ (1), ‘partly true’ (2), or ‘true’ (3) and ending with an open section inviting them to comment more freely on their thoughts and feelings in relation to the intervention.

2.4.6. Program activity

Participants' program activity was registered at the website server in the form of number of completed worksheets, number and duration of log ins, and number of module visits. Therapists kept a record of all contact with adolescents and parents, noting call duration and content cues for each phone call and email throughout the intervention.

2.4.7. A semi-structured interview on user experiences

Qualitative participant feedback was collected by the third and fourth author via semi-structured phone interviews administered at post and follow-up. The interview included components such as website ease of use, parental involvement, motivation, possible treatment barriers, and overall satisfaction with the program.

The following questionnaires were included in the study but not reported here: the Short Moods and Feelings Questionnaire (S-MFQ) (Angold et al., 1995); the Strengths and Difficulties Questionnaire (SDQ) (Goodman et al., 1998; Goodman, 1997); the Self-Efficacy Questionnaire for Children (SEQ) (Muris, 2001); the WHO-5 Well-being Index questionnaire (Topp et al., 2015).

2.5. Analytic strategy

Outcomes were primarily analyzed in the form of change in diagnostic status and severity of anxiety symptoms and anxiety impact on adolescents' life and family as measured by the ADIS-CSR, SCAS and CALIS. For all these measures, we calculated within-group effect sizes at the different data points using Cohen's d for repeated measures, calculated by subtracting the post-mean from the pre-mean divided by the pooled standard deviation of those two means ([Mpre − Mpost] / SDpooled). Due to the small number of participants, we did not make use of significance testing. When dealing with smaller samples, standard deviations may also be unreliable. In the present study they were however generally found to be comparable to or greater than those of a larger trial with a similar population (Arendt et al., 2016).

As questionnaires were obtained through an online data collection platform (SurveyXact) only allowing participants to continue when all questions have been answered, there were no single items missing. In consideration of the small number of participants, we used completer analyses for all quantitative statistics, thus excluding participants who did not complete treatment from all outcome calculations.

3. Results

3.1. Baseline participant characteristics

Baseline participant characteristics are presented in Table 2 and a clinical presentation of each participant at time of inclusion is presented in Table 3. Mean age was 15 years (SD = 1.79) with an equal number of boys and girls. All six participants were living with two biological parents. Parents' mean age was 46.5 (SD = 4.51) and 50.3 (SD = 3.33), mothers and fathers respectively. Four adolescents were diagnosed with social phobia (SoP) as their primary disorder, one with specific phobia (SP), and one with generalized anxiety disorder (GAD). Furthermore, three adolescents suffered from one or two comorbid disorders including GAD, SP, dysthymia and attention deficit hyperactivity disorder (ADHD). Notably, all participants had previously received therapeutic face-to-face treatment in private practices, the length and therapeutic approach of these was, however, not recorded. One participant (Participant 5) dropped out during week five of the 12-week intervention, explaining that she was too busy and lacked the motivation to continue with the program.

Table 2.

Demographic and diagnostic participant characteristics.

Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Participant 6
Age at time of inclusion 13 13 15 15 17 17
Gender ♀ ♂ ♀ ♂ ♀ ♂
On psychopharmacological medication Χ* Χ**
Former therapeutic treatment Χ Χ Χ Χ Χ Χ
Living with two parents Χ Χ Χ Χ Χ Χ
Parent age (m/f) 43/55 51/53 42/46 44/49 53/51 46/48
Parent highest education
 Tertiary degree or higher m/f m m/f m
 Vocational education f f f m/f
 High school m
Household income (Euro)
 >120.000 Χ Χ
 95.000–120.000 Χ Χ
 65.000–94.999 Χ
 <65.000 Χ
Primary diagnosis
 GAD Χ
 SoP Χ Χ Χ Χ
 SP Χ
Comorbid diagnoses
 GAD Χ Χ
 SP Χ
 Dysthymia Χ
 ADHD Χ
Number of anxiety disorders per adolescent 1 4 2 1 1 1
Open in a new tab

Note. *Sertraline; **Concerta; ♂: male; ♀: female; m: mother; f: father; GAD: generalized anxiety disorder; SoP: social phobia; SP: specific phobia; ADHD: attention deficit hyperactivity disorder.

Table 3.

Clinical presentation of participants at time of inclusion.

Participant 1
  • •

    13-year-old girl.

  • •

    Described as sensitive, anxious, and worrisome but also a sociable and popular girl who enjoyed going to school and felt comfortable with her classmates.

  • •

    Attended several different extracurricular activities when not in school, such as tennis, badminton and piano lessons; however, due to her fears and worries Participant 1 met with only one friend outside of school and preferred staying at home.

  • •

    The anxiety was predominantly expressed at night as stomach aches, difficulty sleeping, obsessions about “doing uncomfortable things”, compulsions such as excessive hand washing and counting, and general worries about her own and other's health, and personal appearance and performance.

  • •

    Two years prior to inclusion, Participant 1 was referred to the local Child Psychiatry Unit because of visual hallucinations and suicidal ideations. Her symptoms were seen in the light of a range of stressful events in the nearest family (mental and physical illnesses) and the family was encouraged to seek out psychotherapeutic help. During the following months, Participant 1 saw a school psychologist with moderate effect.
Participant 2
  • •

    14-year-old boy.

  • •

    Described as sociable and friendly with good family relations and positive but limited peer relations.

  • •

    The primary fear centered on vomiting and prevented Participant 2 from many things like going to school, using public transportation and being physically active. Most worries centered on the durability of foods and the risk of getting sick and vomiting, but more general worry themes such as family safety, social situations, and international events also burdened him on an everyday basis.

  • •

    The fears and worries were expressed as constant nausea, panic-like symptoms, thoughts of self-harm or harming parents, sleep- and concentration difficulties, tiredness, and periodic school absenteeism.
Participant 3
  • •

    16-year-old girl.

  • •

    Described as lonely with low self-esteem, dyslexia and limited peer relations. If asked, she preferred staying in her room by herself.

  • •

    Her primary fears centered on social events and made school activities and social gatherings extremely difficult and unpleasant.

  • •

    Participant 3 had a history of self-harming (burns, pinches, etc.) when feeling angry or sad; at time of inclusion she had retained from self-harm for approximately two months.

  • •

    Regularly experienced so-called “dark weeks” characterized by sadness, compensatory eating, feelings of worthlessness and hopelessness, trouble sleeping, and difficulties gathering her thoughts; 7 months prior to inclusion, she wrote a letter to her parents describing suicidal thoughts and was thereafter referred to see a psychologist every other week. At time of inclusion, she did not meet the criteria for major depression.
Participant 4
  • •

    15-year-old boy.

  • •

    Described as sensitive, control- and security-seeking, still preferring sleeping with his parents.

  • •

    Thought of himself as socially able but explained that he favored social interactions in smaller groups and disliked parties.

  • •

    His fears mainly centered on PE classes in which he refused to take part. When asked, he reported sports made his stomach ache, while his parents believed the fear to be related to the changing of clothes or to excessive internal performance standards.

  • •

    Participant 4 found eating difficult and reported frequent stomach aches, also when not being physically active. He did not find his restricted eating to be problematic and when asked presented himself as “very, very, very picky” in relations to food.

  • •

    Three years prior, Participant 4 had been referred to a local outpatient OCD clinic because of hoarding (e.g. garbage and things from nature). He was diagnosed with OCD ‘in remission’ and treated during a subsequent psychotherapeutic course. At time of inclusion, some OCD symptoms had returned such as brushing teeth for 30 min twice a day.
Participant 5
  • •

    17-year-old girl.

  • •

    Described as perfectionistic with high demands to her own performances. Participant 5 explained that she did not thrive among other people and clarified that she felt like lacking a “filter” in social situations, which made it difficult for her to build and maintain face-to-face friendships. Online this was not an issue, and she reported having multiple social relations through the internet.

  • •

    At time of inclusion, Participant 5 had been struggling with anxiety for two years, keeping her from driving with the bus and attending public school, why she was enrolled in a special education program with reduced hours. Her anxiety was mainly expressed as worries concerning social events and possible humiliations, and she found informal and unstructured social gatherings particularly stressful.

  • •

    Two years prior to inclusion, she had undergone clinical assessment at the local Child Psychiatry Unit and was diagnosed with ADHD for which she still received Concerta. The family perceived the ADHD to be in control, but according to Participant 5, it still caused inner turmoil and made it difficult for her to concentrate in class. At time of inclusion, she did not fulfil the criteria for ADHD and had just ended a psychotherapeutic ACT course of 10 sessions.
Participant 6
  • •

    17-year-old boy.

  • •

    Described as anxious and perfectionistic, and suffering from anxiety since the age of 11, although the anxiety had had different “faces” throughout the years.

  • •

    Liked being in school but worried excessively of what others might think of him and was often overwhelmed by homework. He was also unable to ask questions in class and to do school-related presentations, and he avoided all social activities outside school.

  • •

    Although well-liked by his peers, Participant 6 struggled with social relations and preferred to spend time alone or with his twin brother, who was also his best friend.

  • •

    Because of excessive fears and worries, he had been seeing a psychologist six years earlier and was previously assessed at the local Child Psychiatry Unit with no resulting diagnoses or treatments.
Open in a new tab

Note. OCD: obsessive compulsive disorder; ADHD: attention deficit hyperactivity disorder; ACT: acceptance and commitment therapy.

3.2. Feasibility

3.2.1. Program activity

The five adolescents completed a mean of 6.4 modules (SD = 2.51, range 2–8) of the eight modules prescribed. All five participants who completed the intervention chose to accept the offer of extended program access beyond the booster call. Notably, only two participants (Participant 3, Participant 6) logged in to the program web site during prolonged access period, and they did so only once. Table 4 presents an overview of participants' program activity and support for the 12-week intervention period.

Table 4.

Overview of participants' program activity and support.

# modules completed # log ins Mean duration of log ina # calls T/A Mean duration of calls T/A # emails T/A # calls T/P Mean duration of calls T/P # emails T/P
Participant 1 8 18 32.3 13 13.2 3 1 42 3
Participant 2 7 23 18.4 13 13.0 0 2 36 0
Participant 3 2 25 27.8 16 31.1 1 5 22.4 3
Participant 4 8 26 13.1 11 16.6 2 2 11 0
Participant 6 7 23 21.7 11 16.1 0 2 7 0
Mean (SD) 6.4 (2.51) 23 (3.08) 22.66 (7.58) 12.8 (2.05) 18.73 (12.10) 1.2 (1.30) 2.4 (1.52) 23.0 (13.64) 1.2 (1.64)
Open in a new tab

Note. #: number; T/A: between therapist and adolescent; T/P: between therapist and parent.

a

Duration is reported in minutes.

3.2.2. User experiences

All six participants found the Chilled Out program easy to use. Several of them explained that they surfed online daily, making it easy to familiarize with and use Chilled Out. Three participants (Participant 2, Participant 3, Participant 6) expressed the notion that it was not the website itself or the different online exercises that had been difficult, but the fact that they had to expose themselves ‘in real life’ to the things they feared the most:

“I haven't experienced too many difficulties, I think… What I found most challenging was those things you had to do in real life.” (Participant 6).

All participants liked the program being structured around 8 separate modules, thereby making the content more manageable and offering the possibility of revisiting program elements that were at first sight hard to comprehend:

“One of the advantages was that I could choose to go back and listen to things over and over and over again. I could hear them a hundred times without feeling stupid, and I could sit and just keep hearing them, if I wanted to.” (Participant 3).

Other advantages mentioned by adolescents and parents were: (a) the comfort of home; (b) elimination of transportation time and costs; (c) flexibility; (d) anonymity; (e) less shame in comparison to regular courses of therapy.

Asked to name the most prominent treatment barriers, i.e. conditions or circumstances that were perceived by adolescents or parents to have had a negative impact on participants' treatment effect, three factors were mentioned by more than one participant; (a) Time: four adolescents and one parent explained that they had found it difficult to complete the program within the given time frame of 12 weeks. Also, they had been struggling fitting program work and phone calls into their busy schedules; (b) Technical difficulties; four adolescents and one parent reported having experienced technical issues such as slow loading videos or problems logging in to the program; (c) Immaturity: two parents worried that adolescents might not be mature enough to go through the program on their own and ignore the many distractions that they face in their homes and daily lives in general. Example quote:

“The good thing is that you don't have to drive anywhere to do it. You can just sit at home in your own living room. But the bad thing was also, that because you were at home, there were so many other things, tempting things, you could do.” (Participant 2).

3.2.3. Therapeutic support

Participants received a mean of 12.8 therapist phone calls (SD = 2.05; range 11–16) during the 12-week intervention, with an average call duration of 18.73 min (SD = 12.10; range 2–70) (see Table 4 for individual data). One of the participants, Participant 3, had exceptionally long conversations with her therapist (average call duration 31.1 min), partially due to an extremely low self-esteem, a preponderance of negative thoughts and increasingly depressed mood. In retrospect, her treatment came to bear resemblance to an ordinary psychotherapeutic discourse. During the intervention adolescents sent on average 1.2 emails to their therapist (SD = 1.30, range 0–3). Generally, the adolescents expressed great satisfaction with the therapeutic conversations at the feasibility interview post treatment, explaining that the phone calls had been both motivational and helpful in solving challenges that arose during the intervention. These statements were supported by adolescents' scores on the WAI-S, in which they rated their experience of the therapeutic alliance with a mean of 5.98 (SD = 1.13; range 5.0–6.83) on a scale from one to seven, with higher scores indicating better alliances.

Parents received a mean of 2.4 (SD = 1.52, range 1–5) therapist phone calls with an average call duration of 23 min (SD = 13.64, range 3–42). Parent calls were largely focused on (1) adjusting the appropriate amount of parent involvement and (2) helping parents understand program elements and practice tasks. At the feasibility interview post treatment, parents who had been in contact with the therapist rated their interaction as either ‘good’ or ‘very good’. Generally, most parents supported the adolescents' independency working with the program, simultaneously struggling with the fact that they did not know much about the adolescents' program activities and therapeutic progress or the lack hereof:

“Sure, I would have liked to help her more, but it is of no use helping someone, who won't accept your assistance. The way that she feels now, and the kind of person she is, we couldn't have helped her any more than we did. But it sure would have been nice, if we'd been allowed to help her more…” (Participant 3's mother).

When asked how the adolescents perceived their parents' involvement, all responded that they had found it appropriate, elaborating that they had gotten the exact amount of help they had called for.

3.2.4. Treatment satisfaction

According to the experience of service questionnaire completed post treatment, both adolescents and parents were generally satisfied with the intervention. Participant 5 and her mother did not complete the questionnaire. For the statement ‘The intervention helped me/my child’, none of the respondents answered ‘Not true’. Three adolescents (50%), one mother (16.7%), and three fathers (50%) responded ‘True’, while two adolescents (33.3%), four mothers (66.7%), and three fathers (50%) responded ‘Partly true’ to the statement. Example quotes:

“I've received tools and I've had a pleasant experience being in treatment. I haven't felt ashamed of seeing a therapist. I've also become more open than I was before the program started. Now I can tell people, that I am not doing so great and that I suffer from anxiety. I couldn't do that before.” (Participant 3).

“I don't think we've seen the result of this yet; I mean, it's not like a miracle cure saying ‘Boom! Now he can do everything, even walk on water’, because he can't. But he has been given a lot of things he can use, and I've been given a lot of things that I can use. And I really like the way it [the program] works.” (Participant 6's mother).

To the statement ‘If a friend needed this type of help, I would recommend him/her joining the intervention’, three adolescents (50%), four mothers (66.7%), and four fathers (66.7%) answered ‘True’, one adolescent (16.7%), mother (16.7%), and father (16.7%) answered ‘Partly true’, while one adolescent (16.7%), no mothers, and one father (16.7%) responded ‘Not true’. All adolescents and parents answered ‘Not true’ to the statement ‘The intervention made me/my child feel worse’. Two participants (Participant 5, Participant 4) would have preferred regular face-to-face therapy. The remaining four participants preferred the internet-administered treatment modality:

“Well, I've been in therapy several times now, seeing several different therapists […]. I found it much more comfortable using a computer… It was nice not having to sit in front of another person, being interrogated about how you felt.” (Participant 1).

Of the six parents interviewed at post and follow-up, four expressed general satisfaction with the online treatment format, while two (parents of Participant 2 and Participant 4) called for an introductory face-to-face meeting pre-treatment in which they could have met with the therapist and other included families.

3.3. Outcomes

3.3.1. Changes during baseline

Table 5 presents mean scores, standard deviations, and within-group effect sizes (Cohen's d) for all continuous outcome measures from pre-I, pre-II, post-treatment and 3-month follow-up, while Table 6 shows individual diagnoses and associated CSR values from pre-I, pre-II and post.

Table 5.

Mean scores, standard deviations, and effect sizes for all continuous measures.

Pre-I M (SD) Pre-II M (SD) Post M (SD) FUP M (SD) Effect sizesa Pre-I to Pre-II Effect sizesa Pre-II to Post Effect sizesa Post to FUP
ADIS-CSR primary diagnosis 6.40 (1.67) 6.00 (1.58) 3.40 (2.19) – d = 0.25 d = 1.36 –
ADIS-CSR all anxiety diagnoses 5.89 (1.36) 4.56 (2.83) 1.89 (2.37) – d = 0.60 d = 1.02 –
SCAS-C adolescents 30.80 (27.11) 28.40 (23.22) 19.80 (14.41) 23.80 (13.97) d = 0.10 d = 0.45 d = −0.28
SCAS-P mothers 26.60 (10.53) 30.60 (14.93) 18.40 (11.89) 18.80 (12.32) d = −0.31 d = 0.90 d = −0.03
SCAS-P fathers 32.60 (12.42) 32.60 (13.35) 31.80 (16.77) 24.60 (12.74) d = 0.00 d = 0.05 d = 0.48
CALIS adolescents 10.40 (13.99) 9.00 (10.56) 9.20 (9.63) 9.20 (9.28) d = 0.11 d = −0.02 d = 0.00
CALIS mothers 28.60 (8.85) 24.80 (11.48) 14.40 (10.50) 14.60 (14.29) d = 0.37 d = 0.95 d = −0.02
CALIS fathers 26.40 (5.32) 23.00 (6.08) 21.00 (11.92) 17.60 (11.89) d = 0.60 d = 0.21 d = 0.29
Open in a new tab

Note. ADIS: Anxiety Disorder Interview Schedule for DSM-IV; CSR: Clinical Severity Rating (ADIS); SCAS-C/P: Spence Children's Anxiety Scale Child/Parent version; CALIS: Child Anxiety Life Inference Scale.

a

Positive effect sizes indicate improvement.

Table 6.

Diagnostic status and clinical severity ratings.

 Pre-I
 Pre-II
 Post
Diagnosis CSR Diagnosis CSR Diagnosis CSR
Participant 1 GAD* 6 GAD* 5 GAD* 3
Participant 2 SP (throw up)* 8 SP (throw up)* 7 SP (throw up)* 4
SP (dentists) 6 SP (dentists) 5 SP (dentists) 0
SP (needles) 5 SP (needles) 0 SP (needles) 0
GAD 5 GAD 0 GAD 0
Participant 3 SoP* 8 SoP* 8 SoP* 6
GAD 5 GAD 6 GAD 0
Dysthymia 6 Dysthymia 7 Dysthymia 6
Participant 4 SoP* 4 SoP* 4 SoP* 0
Participant 6 SoP* 6 SoP* 6 SoP* 4
Open in a new tab

Note. CSR: Clinical Severity Rating (ADIS); GAD: generalized anxiety disorder; SoP: social phobia; SP: specific phobia; ADHD: attention deficit hyperactivity disorder; *: primary anxiety disorder.

Results from baseline ADIS assessments (pre-I to pre-II) showed small improvements in CSR for two participants (Participant 1, Participant 2) and no change for the remaining three participants for their primary anxiety diagnosis (see Fig. 1). Regarding the sum of all anxiety diagnoses, two participants (Participant 1, Participant 2) showed improvement in CSR, two participants (Participant 4, Participant 6) showed no change in CRS and one (Participant 3) showed a small increase in CSR. One participant (Participant 2) stood out as he experienced a total recovery from two of his anxiety diagnoses during baseline (see Table 6).

Fig. 1.

Fig. 1

Open in a new tab

Clinical Severity Ratings for adolescents' primary anxiety diagnosis at three measuring points.

Results from the self-reported SCAS questionnaire during baseline showed close to no effects for adolescents and fathers (d = 0.10 and 0.00 respectively) while mother reports indicated slight worsening in adolescents' anxiety symptoms with a small effect (d = −0.31). Responses on the CALIS during baseline indicated a decreasing impact of the adolescents' anxiety on their life and family with small to moderate within-group effects (d = 0.11–0.60).

3.3.2. Diagnostic severity

At post assessment, CSR-ratings indicated that all five participants had experienced improvement in symptoms of their primary diagnosis with a large effect size of d = 1.36. The within-group effect size for the sum of all anxiety diagnoses from pre-II to post was also large (d = 1.02). Two participants (Participant 1, Participant 4) no longer met criteria for their primary diagnosis according to ADIS-IV. Additionally, the two participants (Participant 2, Participant 3) who at pre-II had one secondary anxiety diagnosis were free of these at post. Individual CSR-ratings from pre-I, pre-II, and post are depicted in Fig. 1.

3.3.3. Anxiety symptoms and life impact

For the pre-II–post comparison on SCAS, effect sizes were very small for father reports (d = 0.05), close to moderate for adolescent reports (d = 0.45), and large for the mother reports (d = 0.90). Post treatment, reports on CALIS were dispersed with effect sizes indicating that adolescents perceived close to no change in anxiety impact (d = −0.02) while mothers experienced large improvements (d = 0.95) and fathers reported small improvements (d = 0.21).

At follow-up, adolescent reports on SCAS-C indicated a slight worsening in anxiety symptoms with a small within-group effect size (d = −0.28), mother reports demonstrated close to no change (d = −0.03), while father reports indicated improvements with an effect size close to moderate (d = 0.48). Neither the adolescents nor their fathers experienced a change during the follow-up period with respect to the impact of the adolescent's anxiety on their life and family (d = −0.00 and −0.02 respectively), while mothers reported a slight improvement (d = 0.29). Participants' individual SCAS-C scores at all measuring points are depicted in Fig. 2.

Fig. 2.

Fig. 2

Open in a new tab

Adolescents' total scores on Spence Children's Anxiety Scale, Child version at all measuring points.

4. Discussion

The aim of this study was to evaluate the feasibility and preliminary clinical outcome of the Chilled Out program for Danish adolescents with anxiety disorders. One participant dropped out (83% adherence). The remaining completed a mean of 6.5 out of 8 modules. Adolescents and their parents generally expressed satisfaction with the intervention and the majority would recommend it to others, corresponding to what previous studies of similar interventions have found (Spence et al., 2011; Lenhard et al., 2014; Wuthrich et al., 2012). In the qualitative feedback adolescents reported advantages of the therapy format such as elimination of transportation and cost savings, greater anonymity, flexibility, and less shame. Adolescents also concordantly stated that they found it easy to navigate through and understand the program, possibly indicating that the population of 13 to 17-year-olds are indeed ‘digital natives’ (Ho et al., 2016) who will readily and relatively uncritically embrace and grasp this intervention format.

Immediately following treatment, based on the diagnostic interview all six participants demonstrated anxiety symptom reductions in their primary diagnosis, two participants no longer met criteria for their primary anxiety diagnosis (33%) and three participants were free of their secondary anxiety diagnoses (50%). Results from the self-report measures were more dispersed and frequently the adolescent- and parent-ratings did not align. In combination with the small sample size, it is thus difficult to draw any firm conclusions about the preliminary clinical outcome, but the combined results from this study indicate that the guided ICBT program Chilled Out may be a feasible treatment option for adolescents with anxiety disorders.

The most significant barrier mentioned by four of the five participants who finished treatment was finding time to complete the program and the exercises within the given timeframe. In fact, the intervention was perceived as a massive burden to some participants. Although regular therapy may compensate for some of this through the physical presence of a therapist motivating the adolescent and holding him or her accountable for potential procrastination, the perceived lack of time is a common difficulty characterizing many skills-based interventions for children and adolescents carried out in either regular face-to-face format or online (Wuthrich et al., 2012; Cunningham and Wuthrich, 2008; Vigerland et al., 2013).

What could pose as a specific difficulty within online therapy is the reduction of or absence of verbal communication and visual cues between the client and the therapist, making it harder to explore and address therapeutically disruptive factors such as non-compliance, treatment fatigue or resistance. According to participants user experience statements mentioned above, this proved to be a challenge in our study when participants reached the exposure section of the program, inviting them to create exposure hierarchies and facing their biggest fears. Despite detailed descriptions and graphic guidelines in the program combined with individualized problem solving through the weekly phone calls, the development and completion of graduated exposure tasks was difficult for most participants. Challenges related to exposure tasks in ICBT seems though to be a general issue in the literature (see for example Spence et al., 2008) more so than a barrier reserved for the present study. Future studies should focus on designing optimal guidelines for the development of appropriate exposure hierarchies online and examine how to best ensure that the adolescents' hierarchies are feasible.

A few comments on the form and the extent of the therapeutic contact in the present study are needed. When deciding on what type of communication strategy to use, several modalities were discussed including live video, live chat, telephone calls and e-mails. Acknowledging that the program had never been tested in its online version and consequently, that we did not know if it would be effective in reliving anxiety symptoms among the participants, we chose the contact method that we believed would yield the biggest effect while simultaneously providing a good therapeutic alliance between the therapist and the individual participant. As we worried that some participants might not have the necessary hardware or the internet connection required to allow for optimal video transmission, we chose the second most intensive method at hand, i.e. telephone calls. If the Chilled Out program proves effective in relieving adolescents' anxiety symptoms in the upcoming RCT, exploring the potential possibilities of downsizing therapeutic contact while maintaining efficacy should be prioritized.

Worth noticing, most participants highlighted the flexibility of the intervention as a major advantage. At first glance, this emphasizes the immediate suitability of ICBT for adolescents leading very busy lives, but several participants reported finding themselves distracted by competing situations at home, causing them to embark on other more immediately rewarding or less demanding activities, when they were supposed to be working with the program. Of course, all therapeutic interventions necessitate clients to set aside time in their everyday lives whether the format requires face-to-face attendance or not. But studies like the present one indicate that while increasing flexibility, ICBT is likely to simultaneously demand more self-discipline and require clients to take ownership of their treatment to a greater degree than regular face-to-face therapy does it.

The most important limitation of this study is the small sample size as it limits the generalizability of our findings. The study primarily aimed at investigating the feasibility of our own version of the Chilled Out program, but it may also inform other researchers and clinicians of the possible feasibility of ICBT for adolescents with anxiety disorders in general. Furthermore, the study helped accentuate changes and improvements that needed to be implemented into the intervention before evaluation in a forthcoming RCT. Changes will include extension of the intervention time frame from 12 to 14 weeks, the development of a semi-structured manuscript outlining the basic purpose and content of all therapist phone calls, including a photo of the appointed therapist in the information letter that adolescents receive before treatment start to accommodate their expressed feelings of unfamiliarity with the therapist. Also, we've created an online chat-like forum for the adolescents' parents to facilitate exchanges of treatment relevant experiences, questions, good advice, etc. It is our hope that parents will find support and encouragement in this online ‘community’, thereby lowering their experiences of exclusion from treatment. Moreover, hopefully adding to feelings of greater involvement among the parents and accommodating some of the expressed program shortcomings, we have adapted The Chilled Out Mentor Companion that parents receive in the beginning of treatment. Among other things, the parents' role in Chilled Out has been elaborated with a strengthened focus on involvement and motivation of their adolescent. Also, a section advising parents on how to best communicate with and behave towards their adolescent has been added to the Companion. Additionally, we have scheduled a phone call from the adolescent's therapist to the parents within the first two weeks of treatment in which the therapist will introduce him or her-self and make him or her-self familiar to the parents.

5. Conclusion

In conclusion, results from this study indicate that the translated and revised version of the Chilled Out program may be a feasible psychological intervention for Danish adolescents with anxiety disorders. The present evaluation also indicated that ICBT may have the potential to overcome some of the barriers that currently prevent many adolescents from seeking and completing treatment such as greater anonymity, less shame and elimination of transport time and costs. Further trials are needed to thoroughly explore and evaluate the efficacy of ICBT programs and the challenges that arise with this treatment modality, including perceived lack of time, difficulties surrounding exposure tasks and everyday distractions demanding higher motivation and more self-discipline from the adolescents in treatment. This being said, ICBT programs such as Chilled Out may still be a promising solution to many of the treatment barriers faced by this population and thus a direction for improving access to and addressing serious gaps in the current mental health care system.

Acknowledgments

Acknowledgements

The current study was made possible in part by a grant from Trygfonden (grant id: 100886) and from Edith and Godtfred Kirk Christiansens Fond (grant id: 21-5675). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The authors would like to thank our colleagues at Macquarie University for giving us permission to use the program and for supporting the process of translating and revising it. A special thanks to Dr. Lauren McLellan without whom this study would not have been possible. Lastly, we wish to thank the six families that took part in the study and provided us with feedback on the intervention.

Declaration of conflicting interests

The authors declare no conflicting interests.

References

  1. Andersson G. Crc Press; Boca Raton: 2014. The Internet and Cbt: A Clinical Guide. [Google Scholar]
  2. Andersson G. Internet-delivered psychological treatments. Annu. Rev. Clin. Psychol. 2016;12:157–179. doi: 10.1146/annurev-clinpsy-021815-093006. [DOI] [PubMed] [Google Scholar]
  3. Andersson G., Titov N. Advantages and limitations of internet-based interventions for common mental disorders. World Psychiatry. 2014;13:4–11. doi: 10.1002/wps.20083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Angold A., Costello E.J., Messer S.C., Pickles A. Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. Int. J. Methods Psychiatr. Res. 1995;5:237–249. [Google Scholar]
  5. Arendt K., Hougaard E., Thastum M. Psychometric properties of the child and parent versions of Spence children's anxiety scale in a Danish community and clinical sample. J. Anxiety Disord. 2014;28:947–956. doi: 10.1016/j.janxdis.2014.09.021. [DOI] [PubMed] [Google Scholar]
  6. Arendt K., Thastum M., Hougaard E. Efficacy of a Danish version of the cool kids program: a randomized wait-list controlled trial. Acta Psychiatr. Scand. 2016;133(2):109–121. doi: 10.1111/acps.12448. [DOI] [PubMed] [Google Scholar]
  7. Association A.P. Fourth Edition. American Psychiatric Association; Washington, DC: 1994. Diagnostic and Statistical Manual of Mental Disorders. [Google Scholar]
  8. Attride-Stirling J. Commission For Health Improvement; Cambridge, Uk: 2002. Development of Methods to Capture Users' Views of Camhs in Clinical Governance Reviews. [Google Scholar]
  9. Barak A., Klein B., Proudfoot J.G. Defining Internet-supported therapeutic interventions. Ann. Behav. Med. 2009;38:4–17. doi: 10.1007/s12160-009-9130-7. [DOI] [PubMed] [Google Scholar]
  10. Beesdo K., Knappe S., Pine D.S. Anxiety and anxiety disorders in children and adolescents: developmental issues and implications for DSM-V. Psychiatr. Clin. N. Am. 2009;32:483–524. doi: 10.1016/j.psc.2009.06.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Booth M.L., Bernard D., Quine S., Kang M.S., Usherwood T., Alperstein G., Bennett D.L. Access to health care among Australian adolescents young people's perspectives and their sociodemographic distribution. J. Adolesc. Health. 2004;34:97–103. doi: 10.1016/j.jadohealth.2003.06.011. [DOI] [PubMed] [Google Scholar]
  12. Brown A., Ford T., Deighton J., Wolpert M. Satisfaction in child and adolescent mental health services: translating users' feedback into measurement. Adm. Policy Ment. Health Ment. Health Serv. Res. 2014;41:434–446. doi: 10.1007/s10488-012-0433-9. [DOI] [PubMed] [Google Scholar]
  13. Cartwright-Hatton S., Roberts C., Chitsabesan P., Fothergill C., Harrington R. Systematic review of the efficacy of cognitive behaviour therapies for childhood and adolescent anxiety disorders. Br. J. Clin. Psychol. 2004;43:421–436. doi: 10.1348/0144665042388928. [DOI] [PubMed] [Google Scholar]
  14. Christensen H., Batterham P., Calear A. Online interventions for anxiety disorders. Curr. Opin. Psychiatry. 2014;27:7–13. doi: 10.1097/YCO.0000000000000019. [DOI] [PubMed] [Google Scholar]
  15. Collins K.A., Westra H.A., Dozois D.J.A., Burns D.D. Gaps in accessing treatment for anxiety and depression: challenges for the delivery of care. Clin. Psychol. Rev. 2004;24:583–616. doi: 10.1016/j.cpr.2004.06.001. [DOI] [PubMed] [Google Scholar]
  16. Costello E.J., Egger H.L., Copeland W., Erkanli A., Angold A., Silverman Wk F.A. Anxiety Disorders in Children and Adolescents. Ed. 2. Cambridge University Press; Cambridge: 2011. The developmental epidemiology of anxiety disorders: phenomenology, prevalence, and comorbidity. [Google Scholar]
  17. Cunningham M. Overview of design approaches in the cool teens Cd and other computer programs for adolescent anxiety. E-J. Appl. Psychol. 2008;4:6–11. [Google Scholar]
  18. Cunningham M., Wuthrich V. Examination of barriers to treatment and user preferences with computer-based therapy using the cool teens Cd for adolescent anxiety. E-J. Appl. Psychol. 2008;4:12–17. [Google Scholar]
  19. Cunningham M.J., Wuthrich V.M., Rapee R.M., Lyneham H.J., Schniering C.A., Hudson J.L. The cool teens Cd-Rom for anxiety disorders in adolescents: a pilot case series. Eur. Child Adolesc. Psychiatry. 2009;18:125–129. doi: 10.1007/s00787-008-0703-y. [DOI] [PubMed] [Google Scholar]
  20. Davis T.E., Ollendick T.H., Nebel-Schwalm M. Intellectual ability and achievement in anxiety-disordered children: a clarification and extension of the literature. J. Psychopathol. Behav. Assess. 2008;30:43–51. [Google Scholar]
  21. Ebert D.D., Zarski A.C., Christensen H., Stikkelbroek Y., Cuijpers P., Berking M., Riper H. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One. 2015;10 doi: 10.1371/journal.pone.0119895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Elliott B.A., Larson J.T. Adolescents in mid-sized and rural communities: foregone care, perceived barriers, and risk factors. J. Adolesc. Health. 2004;35:303–309. doi: 10.1016/j.jadohealth.2003.09.015. [DOI] [PubMed] [Google Scholar]
  23. Essau C.A., Conradt J., Petermann F. Frequency, comorbidity, and psychosocial impairment of anxiety disorders in German adolescents. J. Anxiety Disord. 2000;14:263–279. doi: 10.1016/s0887-6185(99)00039-0. [DOI] [PubMed] [Google Scholar]
  24. Goodman R. The strengths and difficulties questionnaire: a research note. J. Child Psychol. Psychiatry. 1997;38:581–586. doi: 10.1111/j.1469-7610.1997.tb01545.x. [DOI] [PubMed] [Google Scholar]
  25. Goodman R., Meltzer H., Bailey V. The strengths and difficulties questionnaire: a pilot study on the validity of the self-report version. Eur. Child Adolesc. Psychiatry. 1998;7:125–130. doi: 10.1007/s007870050057. [DOI] [PubMed] [Google Scholar]
  26. Gulliver A., Griffiths K.M., Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. 2010:10. doi: 10.1186/1471-244X-10-113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hedman E., Ljotsson B., Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev. Pharmacoecon. Outcomes Res. 2012;12:745–764. doi: 10.1586/erp.12.67. [DOI] [PubMed] [Google Scholar]
  28. Ho J., Corden M.E., Caccamo L., Tomasino K.N., Duffecy J., Begale M., Mohr D.C. Design and evaluation of a peer network to support adherence to a web-based intervention for adolescents. Internet Interventions. 2016;6:50–56. doi: 10.1016/j.invent.2016.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Holländare F., Gustafsson S.A., Berglind M., Grape F., Carlbring P., Andersson G., Hadjistavropoulos H., Tillfors M. Therapist behaviours in internet-based cognitive behaviour therapy (Icbt) for depressive symptoms. Internet Interventions. 2016;3:1–7. doi: 10.1016/j.invent.2015.11.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Hollis C., Falconer C.J., Martin J.L., Whittington C., Stockton S., Glazebrook C., Davies E.B. Annual research review: digital health interventions for children and young people with mental health problems—a systematic and meta-review. J. Child Psychol. Psychiatry. 2017;58:474–503. doi: 10.1111/jcpp.12663. [DOI] [PubMed] [Google Scholar]
  31. Horvath A.O., Greenberg L.S. Development and validation of the working alliance inventory. J. Couns. Psychol. 1989;36:223–233. [Google Scholar]
  32. James A.C., James G., Cowdrey F.A., Soler A., Choke A. Cognitive behavioural therapy for anxiety disorders in children and adolescents. Cochrane Database Syst. Rev. 2013;Cd004690 doi: 10.1002/14651858.CD004690.pub3. [DOI] [PubMed] [Google Scholar]
  33. Jaycox L.H., Asarnow J.R., Sherbourne C.D., Rea M.M., Laborde A.P., Wells K.B. Adolescent primary care patients' preferences for depression treatment. Adm. Policy Ment. Health Ment. Health Serv. Res. 2006;33:198–207. doi: 10.1007/s10488-006-0033-7. [DOI] [PubMed] [Google Scholar]
  34. Kazdin A.E., Marciano P.L., Whitley M.K. The therapeutic alliance in cognitive-behavioral treatment of children referred for oppositional, aggressive, and antisocial behavior. J. Consult. Clin. Psychol. 2005;73:726–730. doi: 10.1037/0022-006X.73.4.726. [DOI] [PubMed] [Google Scholar]
  35. Kazdin A.E., Whitley M., Marciano P.L. Child-therapist and parent-therapist alliance and therapeutic change in the treatment of children referred for oppositional, aggressive, and antisocial behavior. J. Child Psychol. Psychiatry. 2006;47:436–445. doi: 10.1111/j.1469-7610.2005.01475.x. [DOI] [PubMed] [Google Scholar]
  36. Kessler R.C., Berglund P., Demler O., Jin R., Merikangas K.R., Walters E.E. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Arch. Gen. Psychiatry. 2005;62:593–602. doi: 10.1001/archpsyc.62.6.593. [DOI] [PubMed] [Google Scholar]
  37. Lenhard F., Andersson E., Mataix-Cols D., Ruck C., Vigerland S., Hogstrom J., Hillborg M., Brander G., Ljungstrom M., Ljotsson B., Serlachius E. Therapist-guided, internet-delivered cognitive-behavioral therapy for adolescents with obsessive-compulsive disorder: a randomized controlled trial. J. Am. Acad. Child Adolesc. Psychiatry. 2017;56 doi: 10.1016/j.jaac.2016.09.515. (10–19.E2) [DOI] [PubMed] [Google Scholar]
  38. Lenhard F., Vigerland S., Andersson E., Ruck C., Mataix-Cols D., Thulin U., Ljotsson B., Serlachius E. Internet-delivered cognitive behavior therapy for adolescents with obsessive-compulsive disorder: an open trial. PLoS One. 2014;9 doi: 10.1371/journal.pone.0100773. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Lyneham H.J., Rapee R.M. Agreement between telephone and in-person delivery of a structured interview for anxiety disorders in children. J. Am. Acad. Child Adolesc. Psychiatry. 2005;44:274–282. doi: 10.1097/00004583-200503000-00012. [DOI] [PubMed] [Google Scholar]
  40. Lyneham H.J., Sburlati E.S., Abbott M.J., Rapee R.M., Hudson J.L., Tolin D.F., Carlson S.E. Psychometric properties of the child anxiety life interference scale (CALIS) J. Anxiety Disord. 2013;27:711–719. doi: 10.1016/j.janxdis.2013.09.008. [DOI] [PubMed] [Google Scholar]
  41. Merikangas K.R., He J.-P., Brody D., Fisher P.W., Bourdon K., Koretz D.S. Prevalence and treatment of mental disorders among US children in the 2001–2004 NHANES. Pediatrics. 2010;125:75–81. doi: 10.1542/peds.2008-2598. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Muris P. A brief questionnaire for measuring self-efficacy in youths. J. Psychopathol. Behav. Assess. 2001;23:145–149. [Google Scholar]
  43. Muris P. The pathogenesis of childhood anxiety disorders: considerations from a developmental psychopathology perspective. Int. J. Behav. Dev. 2006;30:5–11. [Google Scholar]
  44. Pennant M.E., Loucas C.E., Whittington C., Creswell C., Fonagy P., Fuggle P., Kelvin R., Naqvi S., Stockton S., Kendall T. Computerised therapies for anxiety and depression in children and young people: a systematic review and meta-analysis. Behav. Res. Ther. 2015;67:1–18. doi: 10.1016/j.brat.2015.01.009. [DOI] [PubMed] [Google Scholar]
  45. Podina I.R., Mogoase C., David D., Szentagotai A., Dobrean A. A meta-analysis on the efficacy of technology mediated CBT for anxious children and adolescents. J. Ration. Emot. Cogn. Behav. Ther. 2016;34:31–50. [Google Scholar]
  46. Reyes-Portillo J.A., Mufson L., Greenhill L.L., Gould M.S., Fisher P.W., Tarlow N., Rynn M.A. Web-based interventions for youth internalizing problems: a systematic review. J. Am. Acad. Child Adolesc. Psychiatry. 2014;53 doi: 10.1016/j.jaac.2014.09.005. (1254–1270.E5) [DOI] [PubMed] [Google Scholar]
  47. Richards D., Richardson T., Timulak L., Mcelvaney J. The efficacy of internet-delivered treatment for generalized anxiety disorder: a systematic review and meta-analysis. Internet Interventions. 2015;2:272–282. [Google Scholar]
  48. Richardson T., Stallard P., Velleman S. Computerised cognitive behavioural therapy for the prevention and treatment of depression and anxiety in children and adolescents: a systematic review. Clin. Child. Fam. Psychol. Rev. 2010;13:275–290. doi: 10.1007/s10567-010-0069-9. [DOI] [PubMed] [Google Scholar]
  49. Rickwood D., Deane F.P., Wilson C.J., Ciarrochi J. Young people's help-seeking for mental health problems. Aejamh (Aust. E-J. Adv. Mental Health) 2005;4 [Google Scholar]
  50. Rickwood D.J., Braithwaite V.A. Social-psychological factors affecting help-seeking for emotional problems. Soc. Sci. Med. 1994;39:563–572. doi: 10.1016/0277-9536(94)90099-x. [DOI] [PubMed] [Google Scholar]
  51. Rickwood D.J., Deane F.P., Wilson C.J. When and how do young people seek professional help for mental health problems? Med. J. Aust. 2007;187:S35–9. doi: 10.5694/j.1326-5377.2007.tb01334.x. [DOI] [PubMed] [Google Scholar]
  52. Ritterband L.M., Gonder-Frederick L.A., Cox D.J., Clifton A.D., West R.W., Borowitz S.M. Internet interventions: in review, in use, and into the future. Prof. Psychol. Res. Pract. 2003;34:527–534. [Google Scholar]
  53. Scahill L., Riddle M.A., Mcswiggin-Hardin M., Ort S.I., King R.A., Goodman W.K., Cicchetti D., Leckman J.F. Children's Yale-Brown obsessive compulsive scale: reliability and validity. J. Am. Acad. Child Adolesc. Psychiatry. 1997;36:844–852. doi: 10.1097/00004583-199706000-00023. [DOI] [PubMed] [Google Scholar]
  54. Silverman W.K., Albano A.M. Psychological Corporation; San Antonio, Tx: 1996. Anxiety Disorders Interview Schedule for Children for DSM–IV: Child and Parent Versions. [Google Scholar]
  55. Silverman W.K., Saavedra L.M., Pina A.A. Test-retest reliability of anxiety symptoms and diagnoses with anxiety disorders interview schedule for DSM-IV: child and parent versions. J. Am. Acad. Child Adolesc. Psychiatry. 2001;40:937–944. doi: 10.1097/00004583-200108000-00016. [DOI] [PubMed] [Google Scholar]
  56. Spek V., Cuijpers P., Nyklíček I., Riper H., Keyzer J., Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol. Med. 2007;37:319–328. doi: 10.1017/S0033291706008944. [DOI] [PubMed] [Google Scholar]
  57. Spence S.H. A measure of anxiety symptoms among children. Behav. Res. Ther. 1998;36:545–566. doi: 10.1016/s0005-7967(98)00034-5. [DOI] [PubMed] [Google Scholar]
  58. Spence S.H., Donovan C.L., March S., Gamble A., Anderson R., Prosser S., Kercher A., Kenardy J. Online CBT in the treatment of child and adolescent anxiety disorders: issues in the development of brave-online and two case illustrations. Behav. Cogn. Psychother. 2008;36:411–430. [Google Scholar]
  59. Spence S.H., Donovan C.L., March S., Gamble A., Anderson R.E., Prosser S., Kenardy J. A randomized controlled trial of online versus clinic-based CBT for adolescent anxiety. J. Consult. Clin. Psychol. 2011;79:629–642. doi: 10.1037/a0024512. [DOI] [PubMed] [Google Scholar]
  60. Stallard P., Velleman S., Richardson T. Computer use and attitudes towards computerised therapy amongst young people and parents attending child and adolescent mental health services. Child Adolesc. Mental Health. 2010;15:80–84. doi: 10.1111/j.1475-3588.2009.00540.x. [DOI] [PubMed] [Google Scholar]
  61. Stasiak K., Fleming T., Lucassen M.F.G., Shepherd M.J., Whittaker R., Merry S.N. Computer-based and online therapy for depression and anxiety in children and adolescents. J. Child Adolesc. Psychopharmacol. 2016;26:235–245. doi: 10.1089/cap.2015.0029. [DOI] [PubMed] [Google Scholar]
  62. Subrahmanyam K., Lin G. Adolescents on the net: Internet use and well-being. Adolescence. 2007;42:659–677. [PubMed] [Google Scholar]
  63. Tillett J. Adolescents and informed consent: ethical and legal issues. J. Perinat. Neonatal Nurs. 2005;19:112–121. doi: 10.1097/00005237-200504000-00007. [DOI] [PubMed] [Google Scholar]
  64. Tillfors M., Andersson G., Ekselius L., Furmark T., Lewenhaupt S., Karlsson A., Carlbring P. A randomized trial of internet-delivered treatment for social anxiety disorder in high school students. Cogn. Behav. Ther. 2011;40:147–157. doi: 10.1080/16506073.2011.555486. [DOI] [PubMed] [Google Scholar]
  65. Topp C.W., Østergaard S.D., Søndergaard S., Bech P. The WHO-5 well-being index: a systematic review of the literature. Psychother. Psychosom. 2015;84:167–176. doi: 10.1159/000376585. [DOI] [PubMed] [Google Scholar]
  66. Tracey T.J., Kokotovic A.M. Factor structure of the working alliance inventory. Psychol. Assess. 1989;1:207–210. [Google Scholar]
  67. Vigerland S., Thulin U., Ljótsson B., Svirsky L., Öst L.-G., Lindefors N., Andersson G., Serlachius E. Internet-delivered CBT for children with specific phobia: a pilot study. Cogn. Behav. Ther. 2013;42:303–314. doi: 10.1080/16506073.2013.844201. [DOI] [PubMed] [Google Scholar]
  68. Wang P.S., Aguilar-Gaxiola S., Alonso J., Angermeyer M.C., Borges G., Bromet E.J., Bruffaerts R., De Girolamo G., De Graaf R., Gureje O., Haro J.M., Karam E.G., Kessler R.C., Kovess V., Lane M.C., Lee S., Levinson D., Ono Y., Petukhova M., Posada-Villa J., Seedat S., Wells J.E. Use of mental health services for anxiety, mood, and substance disorders in 17 countries in the who world mental health surveys. Lancet. 2007;370:841–850. doi: 10.1016/S0140-6736(07)61414-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Wilson C.J., Bushnell J.A., Caputi P. Early access and help seeking: practice implications and new initiatives. Early Interv. Psychiatry. 2011;5(Suppl. 1):34–39. doi: 10.1111/j.1751-7893.2010.00238.x. [DOI] [PubMed] [Google Scholar]
  70. Woodward L.J., Fergusson D.M. Life course outcomes of young people with anxiety disorders in adolescence. J. Am. Acad. Child Adolesc. Psychiatry. 2001;40:1086–1093. doi: 10.1097/00004583-200109000-00018. [DOI] [PubMed] [Google Scholar]
  71. Wuthrich V.M. Centre For Emotional Health, Macquarie University; Sydney: 2014. Chilled Out Mentor Companion. [Google Scholar]
  72. Wuthrich V.M., Cunningham M.J., Rapee R.M. The Wiley-Blackwell Handbook of the Treatment of Childhood and Adolescent Anxiety. John Wiley & Sons, Ltd.; 2013. Cool teens: a computerized intervention for anxious adolescents. [Google Scholar]
  73. Wuthrich V.M., Rapee R.M., Cunningham M.J., Lyneham H.J., Hudson J.L., Schniering C.A. A randomized controlled trial of the cool teens CD-ROM computerized program for adolescent anxiety. J. Am. Acad. Child Adolesc. Psychiatry. 2012;51:261–270. doi: 10.1016/j.jaac.2011.12.002. [DOI] [PubMed] [Google Scholar]
  74. Ye X., Bapuji S.B., Winters S.E., Struthers A., Raynard M., Metge C., Kreindler S.A., Charette C.J., Lemaire J.A., Synyshyn M., Sutherland K. Effectiveness of internet-based interventions for children, youth, and young adults with anxiety and/or depression: a systematic review and meta-analysis. BMC Health Serv. Res. 2014;14:313. doi: 10.1186/1472-6963-14-313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Zachrisson H.D., Rodje K., Mykletun A. Utilization of health services in relation to mental health problems in adolescents: a population based survey. BMC Public Health. 2006;6:34. doi: 10.1186/1471-2458-6-34. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES