Effectiveness of and processes related to internet-delivered acceptance and commitment therapy for adolescents with anxiety disorders: a randomized controlled trial

Abstract

Early access to evidence-based help is crucial for adolescents with anxiety disorders. Internet-delivered acceptance and commitment therapy (iACT) may offer adolescents increased access to care and more flexibility in engaging with treatment when and how they prefer. Process-based therapies, such as ACT, focus on theoretically derived and empirically tested key mechanisms in treatment that enable change. This study aimed to investigate the effectiveness of iACT for adolescents with anxiety disorders. The study also assessed the relationship between psychological flexibility and treatment outcomes and the relationship between participating adolescents’ and therapists’ perceived alliance and treatment outcomes. This was a randomized controlled trial comparing a 10-week intervention group with a wait-list control group. The 52 participants, aged 15 to 19, were recruited from all over Sweden. The treatment was effective in increasing quality of life and psychological flexibility, with moderate between-group effect sizes based on observed values. Changes in psychological flexibility was associated with changes in anxiety symptoms. The results further showed a statistically significant between-group difference in post-treatment diagnoses. No significant time per group interaction was found for anxiety symptoms, as both groups improved. Working alliance was rated as high by both participating adolescents and therapists but showed no significant relationship with treatment outcomes. Participants found the treatment an acceptable intervention. This study shows promising results for iACT in treating adolescents with anxiety disorders. The results suggest the model of psychological flexibility as an important process of change in treatment outcomes. Future research should validate these findings in larger samples and clinical contexts.

Introduction

Anxiety disorders are the most common mental disorders. Up to 1 in 3 individuals suffer from an anxiety disorder during their lifetime, according to compilations of population-based surveys (Bandelow & Michaelis, 2015). Anxiety disorders have the earliest onset of all mental health problems and often debut during childhood or adolescence, with a median onset age of 11 years old (Kessler et al., 2005). Adolescence is a period of intense developmental change, where biological and social factors interact and critical academic and social responsibilities, such as completing one’s education, obtaining a job, or building one’s social network, are acquired and much of the foundation for early adulthood is laid. Untreated anxiety disorders in adolescence are associated with poorer psychosocial functioning in terms of, for example, income and unemployment, maladjustment, chronic stress and poor coping skills, and other forms of psychopathology, including depression in adulthood (Essau et al., 2014). This calls for early access to help for youth with anxiety problems.

Internet-delivered psychological treatment may increase access to evidence-based care by reducing some of the barriers that exist in health care, such as limited therapist time, long waiting times for treatment, geographical inequalities in access to care, and inflexibility in when and where to access treatment (Holmes et al., 2018). Several meta-analyses and systematic reviews have shown evidence for the effectiveness of therapistguided internet-delivered cognitive behavioral therapy (ICBT) for anxiety in adults (Andrews et al., 2018; Kumar et al., 2017; Olthuis et al., 2016). Some meta-analyses have also shown promising results for the ICBT efficacy for children and adolescents (Cervin & Lundgren, 2022; Ebert et al., 2015; Vigerland et al., 2016). Vigerland et al. (2016) included 24 studies (N=1882) on the ICBT effect on children and adolescents for 11 different conditions, both psychiatric and somatic. The results showed moderate between-group effect sizes for treatment groups and wait-listed controls [g=0.62, 95% confidence interval (CI): 0.41, 0.84). Ebert et al. (2015) included 13 studies (N=796) evaluating ICBT for anxiety and depression in youths aged ≤25 years and found an overall between-group effect size for ICBT of g=0.72 (95% CI: 0.55, 0.90) compared with nonactive controls. A recent systematic review included 9 randomized controlled trial (RCT)-studies where patients over 18 years old with pediatric anxiety were assessed pre- and post-treatment using a structured diagnostic interview, as opposed to solely self-report measures (Cervin & Lundgren, 2022). ICBT showed a moderate effect on the remission of primary anxiety diagnosis and functioning as rated by clinicians. Less evidence was found for rated anxiety symptoms, where a small significant effect was found for parent/caregiver-reported anxiety but no significant effect on anxiety was rated by the youths themselves (Cervin & Lundgren, 2022).

Other than just knowing that a certain treatment method works, mediation studies provide evidence that key mechanisms in therapy produce clinical change and are important for optimizing clinical practice and treatment response (Kazdin, 2008). Acceptance and commitment therapy (ACT) is part of a new generation of third waves cognitive behavioral therapies and has a well-developed theoretical foundation and clear theories on what treatment mechanisms enable change. It focuses on stimulating processes relevant to growth and development. Face-to-face ACT has been shown an effective treatment method for anxiety disorders in adults and youths (Fang & Ding, 2020; Gloster et al., 2020; Petersen et al., 2022). ACT delivered through the internet (iACT) shows promising results for adults with generalized anxiety disorders, social anxiety disorders, and health-related anxiety, although it needs to be further evaluated for other anxiety-related disorders and compared to active control groups (Kelson et al., 2019). However, less evidence exists on iACT for children and adolescents.

The primary change mechanism and goal of any ACT treatment are to improve psychological flexibility, defined as the ability to flexible behavior based on situational demands and in accordance with personal long-term values (Hayes et al., 2012). In a recent systematic review, general support was found for the model of psychological flexibility as a mediator in treatment outcomes for anxiety and depression, quality of life, behavioral changes, and functioning (Stockton et al., 2019). However, the included studies had methodological limitations and examined only some concepts of the psychological flexibility model (Stockton et al., 2019). Higher psychological flexibility is associated with better mental health and well-being whereas psychological inflexibility is associated with lower functioning, psychological distress, and rumination (Gloster et al., 2017; Stabbe et al., 2019). The ACT model is also used with youths. Psychological flexibility, conceptualized as acceptance and cognitive defusion and measured by the avoidance and fusion questionnaire for youth (AFQ-Y8), has been found to mediate anxiety outcomes in an out-patient group of adolescents aged 12 to 17 in one study (Swain et al., 2015), and psychological inflexibility degree has been linked to anxiety symptoms (Lønfeldt et al., 2017).

Alliance is another well-studied mechanism of change that has consistently shown a moderate association with outcomes in face-to-face therapy in both adults and youths (Flückiger et al., 2018; Karver et al., 2018). A recently published meta-analysis found that therapeutic alliance is improved when patients receive a preferred treatment (Windle et al., 2020), which makes alliance important to explore in terms of personalizing digital care. A positive alliance in child psychotherapy may affect treatment motivation and provide a positive relational experience (Nuñez et al., 2022). However, the predictive relationship of alliance in digital treatment is less studied and the results are contradictory (Andersson et al., 2012; Andersson & Titov, 2014; Hadjistavropoulos et al., 2017; Vernmark et al., 2019). It is not obvious whether the alliance has equal importance in internet-based therapy, as the internet therapist generally spends less time on the patient as the treatment is largely based on self-help. Moreover, the interaction is mostly in written format through secure asynchronous messages in the treatment platform. Young people are used to communicating and forming relationships over the internet. Alliance as a bi-directional activity between therapists and patients has, however, not yet been evaluated in iACT for adolescents.

Objectives

In summary, access to evidence-based care is crucial for adolescents with anxiety disorders, but such access is limited worldwide and in the Swedish healthcare system. Internet-delivered psychological treatment may provide adolescents with increased access to care and flexibility in when and how they engage in the treatment. Studies have shown promising results for ICBT in adolescents; however, few studies exist on iACT programs for adolescents. Additionally, more research is needed to understand the mechanisms of change that are crucial for positive treatment outcomes. This study aimed to assess the effectiveness of an iACT treatment for adolescents with anxiety disorders. The study also aimed to assess the relationship between psychological flexibility and treatment outcomes and to examine the relationship between participating adolescents’ and therapists’ perceived alliance and treatment outcomes.

Methods

Design

This RCT included 52 participants randomized either to the intervention group, who received a 10-week ACT treatment through the internet, or to a wait-listed control group. The study was approved by the Swedish Ethical Review Authority (2019-05-784).

Participants

Participants were adolescents from all over Sweden. Inclusion criteria were i) meeting criteria for one or more of the anxiety diagnoses (panic syndrome, agoraphobia, social anxiety disorder, and generalized anxiety disorder) or being considered to suffer significant, though subclinical, symptoms within the anxiety spectrum; ii) being aged from 15 to 19 years; iii) speaking and understanding Swedish adequately; iv) having access to a computer, phone, or tablet with internet connection. Exclusion criteria were i) being in ongoing psychological treatment; ii) having recently started or made major changes in psychopharmacological treatment; iii) having a severe mental health or complex psychiatric comorbidity, alcohol or substance abuse, or risk of suicide. Examples of severe mental health as defined by the diagnostic interview with mini-international neuropsychiatric interview for children and adolescents (MINIKID) included severe depressive episode, manic/hypomanic episode, post-traumatic stress disorder, psychotic symptoms, and affective syndrome with psychotic symptoms, anorexia nervosa, bulimia nervosa or binge eating disorder, and antisocial personality disorder.

Procedure

Participants were recruited through advertisements on social media, in schools, primary healthcare centers, and outpatient psychiatric clinics during December 2019 and January 2020. More information and an invitation to be considered for the study were provided on a project-specific website. In the first step of the sample’s pre-assessment, those interested were contacted by email and given a link to a website where they were given study information and could give their consent to participate and fill in measurement forms assessing symptoms of anxiety, quality of life, and psychological flexibility. The forms were estimated to take about 10 minutes to fill in, collected certain demographic data, such as housing situation, gender, and age, and included a measure to screen for severe depression or suicidal ideation. Informed consent was obtained from the adolescent themselves since they all had reached the age of 15, which within the context of research projects, as well as in Swedish healthcare, is considered an age when children can decide themselves whether they want to participate in a treatment study or certain care interventions. However, it was emphasized, for example in the study information to the participants, that the adolescent obviously was welcome to inform their guardians if they desired.

Those who met the eligibility criteria were contacted by phone and underwent a diagnostic interview using the MINI-KID instrument (Sheehan et al., 2010). The interviews lasted an average of about 60 minutes each. The diagnostic interviews were conducted by 4 students in their last semester of the master’s program in psychology, all of whom had had training in clinical patient work and CBT during their education. The student therapists received regular supervision from a clinical psychologist who worked at the participating clinic Psykologpartners, and who was available to answer questions about their assessments and clinical work with participants. The supervising psychologist had training in CBT and a couple of years of experience working with ICBT. Two of the students (AKE & SS) and the supervising clinical psychologist (ER) are co-authors of the present study.

During the diagnostic interview, 7 participants were found to not meet the diagnostic criteria for an anxiety diagnosis. These subclinical participants had signed up for the research study on equal terms as the other participants and on the basis that they experienced problems due to their anxiety, and thus a decision was made to include them in the study. Clinical and subclinical participants did not differ on demographic variables, the main outcome measure for anxiety [Spence children’s anxiety scale-short version (SCAS-S)], or quality of life at the pretreatment assessment. However, subclinical participants had higher psychological flexibility (lower scores on the AFQ-; p=.050) and lower scores on the secondary outcome measure for anxiety [generalized anxiety disorder 7-item (GAD-7); p=.047] at the pre-treatment assessment.

Eligible participants were randomized to either an intervention group or a wait-listed control group. The researchers responsible for randomization used the tool random.org. In another randomization, each participant was assigned to a different therapist from the one who conducted the MINI-KID diagnostic interview. After randomization, the intervention group was given access to the 10-week treatment program. The treatment period went from February to April 2020. Participants in the intervention group received weekly feedback from their therapist during the treatment through asynchronous messages on the treatment portal. They were also offered treatment support in a start-up call with their therapist and subsequent calls in the middle and at the end of the treatment.

The participants in the control group were placed on a waiting list during the study period. The participants in the control group were able to contact the responsible researchers if they began to deteriorate in their mental health and would, in that case, be referred to regular care. By the end of the 10-week study period, participants in the control group were contacted via email or phone calls and were offered the so-called on-demand treatment: they were given access to the treatment program but without active weekly therapist support. However, they could contact the study therapists with specific questions about the program.

At the end of the treatment period, all participants were sent an email link to post-treatment measurement forms. Participants in the intervention group also answered questions about their treatment experience through the online link. All participants diagnosed with at least one anxiety disorder at the pre-treatment assessment were contacted by telephone at the end of the treatment period and completed another diagnostic interview with the MINI-KID instrument. Only the diagnoses confirmed at baseline were screened for again. The telephone interview lasted about 10 minutes and was conducted by the same therapist who conducted the interview before the start of treatment. Participants with subclinical problems only answered self-assessment forms posttreatment and did not go through the final diagnostic interview. After the post-treatment assessment, participants in the control group were offered access to the treatment program, but with no active treatment support. Figure 1 shows the CONSORT flowchart of the study, while Table 1 shows the demographic variables of the participants in the study.

Intervention

The research participants in the intervention group were treated for 10 weeks with the treatment program, Ångesthjälpen Ung (Anxiety Help for Adolescents; https://www.kbtonline.se; https://angesthjalpen-ung.webnode.se/), developed by the private psychology company Psykologpartners. Anxiety Help for Adolescents is an evidence-based transdiagnostic internet treatment based on ACT and CBT interventions to treat anxiety in young people (Fang & Ding, 2020; Gloster et al., 2020; Reynolds et al., 2012). The program is based on the ACT treatment structure designed by Hayes et al. (2012), with modifications specifically focused on anxiety (Eifert & Forsyth, 2005) and adapted to suit young people in accordance with the work of Bailey et al. (2012) and Hayes & Ciarrochi (2015). The content, overall structure, language level, concretization of theoretical concepts, and clinical examples in the program have been adapted to suit the age and maturity of adolescents and young adults. To motivate young people to work with the program, its design includes a high degree of interactivity, a variety of ways to convey treatment content, and a limited amount of text. The program comprises eight modules consisting of textbased information, video and audio clips, exercises, and homework assignments. The program is designed for use with the guidance of a therapist, is available within the healthcare system in large parts of Sweden, and is also accessible for private use through the company Psykologpartners. No formal evaluations or research studies on the program have yet been published.

Figure 1.

CONSORT flow diagram.

Participants in the intervention group were supported with weekly written feedback from one of the four participating master students in psychology for this study, all of whom had training in clinical patient work. The student therapists received regular supervision from the clinical psychologist who worked at Psykologpartners (ER). If participants stopped working with the treatment, the therapists contacted them via messages through the treatment program, email, or telephone call to increase their motivation and compliance. Table 2 shows the content of the ACT treatment program.

Measurements

MINI-KID (Sheehan et al., 2010) is a clinical semi-structured diagnostic interview used to assess the prevalence of 16 common psychiatric diagnoses in childhood and suicidality. The interview has been found to be a valid and reliable interview in both clinical samples and the general population (Duncan et al., 2018). It has adequate test-retest reliability, discriminant validity, and convergent validity and is comparable to other well-used standardized diagnostic interviews. However, agreement on disorders between parents and youths is low (Duncan et al., 2018).

SCAS-S is a well-established self-assessment form to assess children’s symptoms of the diagnoses of generalized anxiety disorder, social anxiety, panic syndrome, agoraphobia, obsessive- compulsive disorder, and fear of physical injury (Essau et al., 2011). The Swedish version of the instrument has shown good psychometric properties (Olofsdotter et al., 2015) as has the short form which was used for this study (Ahlen et al., 2018). Cronbach’s a in this study was .76.

GAD-7 is a well-established self-assessment form that measures symptoms of worry (Spitzer et al., 2006). There are thresholds for mild, moderate, and severe anxiety. GAD-7 has shown good psychometric properties in adults (Rutter & Brown, 2017; Spitzer et al., 2006). The measurement scale was evaluated in a large population study of Finnish adolescents aged 14 to 18 years and indicated good psychometric properties for adolescents (Tiirikainen et al., 2019). Cronbach’s a in this study was .82.

Table 1.

Demographic variables of participants.

Variable		Intervention group (n=27)	Control group (n=25)
Age, mean (SD)		16.63 (1.39)	16.60 (1.04)
Gender, frequencies (%)	Woman	23 (85.2)	20 (80.0)
	Man	4 (14.8)	5 (20.0)
	Other	0 (0)	0 (0)
Living situation, frequencies (%)	Lives with someone from the original family	24 (88.9)	23 (92.0)
	Lives with another adult	1 (3.7)	0 (0)
	Lives with a friend	0 (0)	0 (0)
	Lives with partner	0 (0)	2 (8.0)
	Lives alone	2 (7.4)	0 (0)
Occupation	Studying - primary school	9 (33.3)	4 (16.0)
	Studying - high school	14 (51.9)	20 (80.0)
	Studying - post-secondary education	1 (3.7)	0 (0)
	Studying - university	0 (0)	0 (0)
	Working	3 (11.1)	1 (4.0)
	Unemployed	0 (0)	0 (0)
	Sick leave	0 (0)	0 (0)
Highest completed education, frequencies (%)	Not yet finished primary school	9 (33.3)	4 (16.0)
	Primary school	14 (51.9)	20 (80.0)
	High school	4 (14.8)	1 (4.0)
	Post-secondary education	0 (0)	0 (0)
	University	0 (0)	0 (0)
Current psychopharmacological medication,	Regularly	1 (3.7)	0 (0)
frequencies (%)	When necessary	1 (3.7)	2 (8.0)
	Both regularly and when necessary	0 (0)	0 (0)
	None	25 (92.6)	23 (92.0)
Previous psychological treatment, frequencies (%)	Yes	17 (63.0)	11 (44.0)
	No	9 (33.3)	14 (56.0)
	Uncertain	1 (3.7)	0 (0)
Prevalence of specific anxiety syndromes at	Panic syndrome	9 (33.3)	8 (32.0)
pre-assessment,a,b frequencies (%)	Agoraphobia	15 (55.6)	14 (56.0)
	Social anxiety disorder	14 (51.9)	12 (48.0)
	Generalized anxiety disorder	15 (55.6)	11 (44.0)
Number of diagnoses at pre-assessment,a	0	3 (11.1)	4 (16.0)
frequencies (%)	1	9 (33.3)	7 (28.0)
	2	6 (22.2)	6 (24.0)
	3	4 (14.8)	6 (24.0)
	4	5 (18.5)	2 (8.0)

^aAssessment of diagnoses based on the mini-international neuropsychiatric interview for children and adolescents, version 6.0; bParticipants could meet the criteria for more than one diagnosis. SD, standard deviation.

Brunnsviken brief quality of life scale (BBQ) is a self-assessment form that measures the perceived quality of life based on six different areas of life. It has shown good psychometric properties and a cut-off value has been proposed to differentiate between clinical and non-clinical populations (Lindner et al., 2016). BBQ was previously used in a study evaluating a selfhelp intervention based on ACT for overweight people (Wallin et al., 2018). Cronbach’s a in this study was .71.

AFQ-Y8 is a self-assessment form based on the ACT model of mental health. It was developed for adolescents and measures psychological inflexibility in terms of acceptance and cognitive defusion (Greco et al., 2008). Higher scores indicate higher psychological inflexibility and lower scores indicate psychological flexibility. Thus, lower scores are desirable. The measurement has obtained good psychometric properties in a school-based sample of 329 children with internal reliability of α=.83, support for convergent and construct validity and a confirmatory factor analysis supporting a one-factor model of acceptance and defusion (Greco et al., 2008). The Swedish version has been evaluated among adolescents aged 15 to 20 years and has shown good psychometric properties (Livheim et al., 2016). Cronbach’s a in the present study was .71.

Working alliance inventory-short revised (WAI-SR; Hatcher & Gillaspy, 2006) and working alliance inventory-short (WAI-S; Tracey & Kokotovic, 1989) are self-assessment forms developed to measure the three factors in Bordin's model for working alliance; bond, task and goals (Bordin, 1983). The forms measure the strength of the working alliance in therapy between the therapist and the patient. In this study, WAI-SR was used to measure alliance at week 3 and week 7 in the treatment and WAI-S was used to measure expected alliance. The latter was slightly modified by reformulating the questions from present to future tense. A similar modified version has previously been used in a study by Bergman Nordgren et al. (2013). Both WAI-SR and WAI-S are available as a patient and therapist version and consist of 12 and 10 items respectively (Hatcher & Gillaspy, 2006; Tracey & Kokotovic, 1989). The answers are measured on a 5-point Likert scale in the WAI-SR with scores ranging from 12-60 in the patient version and 10-50 in the therapist version. For WAI-S, the answers are instead measured on a 7-point Likert scale, with scores ranging from 12-85 in the patient version and 10-70 in the therapist version. WAI-SR has shown good internal consistency in both outpatient (α=.93) and inpatient care (α=.90) and has demonstrated convergent validity with another alliance measurement (Munder et al., 2010). Cronbach’s α for patient rated-rated alliance in this study was .83 for expected alliance, .78 for alliance measured at week 3 and .83 for alliance measured at week 7. Cronbach’s α for the corresponding therapist ratings of alliance was α=.55 for expected alliance, α=.95 for alliance measured at week 3, and α=.96 for alliance measured at week 7.

Patient health questionnaire-9 (PHQ-9; Kroenke & Spitzer, 2002) is a self-assessment form that measures symptoms of major depression as well as symptom severity. Each question corresponds to a diagnostic criterion for major depression in the diagnostic and statistical manual of mental disorders (DSM). The instrument showed excellent internal reliability (Cronbach’s α of 0.89) and test-retest reliability as well as good construct validity in the original validation study in a primary care sample (Kroenke et al., 2001). Meta-analytic results show that PHQ-9 is a valid tool to correctly diagnose major depression disorder (Gilbody et al., 2007). In this study, PHQ-9 was used solely to screen for severe depression and suicidal ideation before inclusion in the study. Cronbach's α in this study was .78.

Table 2.

Modules in the treatment program.

Module	Title of module	Core treatment component	Content, focus and example of exercises
1 What is anxiety?		Psychoeducation	Focused on gaining knowledge about anxiety. Exercises to identify triggers for anxiety
2	What’s important for you	Values	Focused on contacting meaningful experiences in life. Exercises include writing down important qualities in life and activities associated with vitality, as well as identifying barriers, avoidance behaviors, and meaningful coping strategies. Exercises also include setting goals for the treatment
3	Why did you end up here?	Experiential avoidance and functional analysis	Focused on the function of behaviors in different contexts. Writing exercises to learn to discriminate experiential avoidance and analyze behaviors in everyday life by registrations.
4	I don’t dare but I will do it anyway	Acceptance and exposure	Presenting an alternative to avoidance of feelings and thoughts. Focus on acceptance as a tool to start moving in one’s valued direction. Exercises include writing down actions in line with values and committing to action by starting to expose oneself to anxiety-provoking situations
5	Thought zoom helps you forward	Cognitive defusion	Focused on defusion from negative thoughts by seeing them as something the mind has produced rather than absolute truths. Apply cognitive defusion as a means for committed action. Exercises include continuing exposure to anxiety-provoking situations
6	Be present	Mindfulness	Focused on learning to be present here and now. Exercises include performing mindfulness exercises
7	Walk towards your sun	Summary of previous modules	Summing up and a focus on continuing to behave in the service of chosen values. Exercises include follow-up on goals for treatment and planning new exposure-exercises in line with valued direction
8	Make a plan for the future	Maintenance	Focused on what the participant has achieved during treatment. Exercises include planning for the future and strategies to deal with setbacks

Treatment acceptability includes patient beliefs about the treatment’s reasonability and appropriateness and their satisfaction with it. Treatment acceptability was assessed by the question: Would you recommend the treatment to a friend with similar problems? In addition, the participants were asked which interventions in the treatment program they had found most helpful.

Data analysis

Effects of internet-delivered acceptance and commitment therapy

The data analyses were performed in IBM SPSS Statistics 28 (Armonk, New York, USA). Initial analyses were performed with chi-squared and t-tests (and were appropriate Fisher exact test) to find any statistically significant differences in demographic variables, number of fulfilled diagnoses, and outcome measures at the pre-treatment assessment between the two treatment conditions, the clinical and subclinical participants, and participants who completed the self-assessment scales at all measurement points and those lost to follow-up.

Main analyses for outcome measures for anxiety, quality of life, and psychological flexibility were based on the intention to treat by analyzing all participants that were randomized and used multilevel modelling (MLM) for repeated measures, which were fitted with full information maximum likelihood estimation and an unstructured covariance structure. MLM was chosen to handle the dependency in the data since several observations were made for the same individuals and because it is an advantageous method to deal with missing data. The analyses tested the overall change over time in the intervention group as well as the interaction between the two groups and time, where time was included as a factor (pre- and post-intervention). We used a model with fixed effects for time, treatment condition and time-by-treatment interaction effect. The time slope was modeled as a fixed effect due to the low number of data observations which hampered the computation of random slopes, and since significance test determined by -2LL did not show that a model with random effects for time was adequate. Within-group (Cohen’s d_{RM, pool)} and between-group effect sizes (Cohen's d_S) were calculated on observed data (Cohen, 2013). Effect sizes above d=.20 were interpreted as small, above d=.50 as medium, and above d=.80 as strong.

Complementary analyses for the main analyses were made based on repeated measures mixed analysis of variance (ANOVA), using both last observations carried forward to deal with missing data as well as listwise deletion for analyses per protocol. Results from these analyses did not differ significantly from the LMM analyses. Results from the ANOVAs are found in Appendix I (Supplementary Material).

The effect of the treatment on diagnoses remission was assessed using a chi-squared test of between-group differences in post-treatment diagnoses.

In addition, further analyses assessed hypothesized key mechanisms in treatment. The relationship between psychological flexibility and severity of anxiety symptoms was assessed by comparing changes in psychological flexibility and changes in anxiety from pre- to post-assessment using Spearman’s Rho coefficients (r_s), where correlations above r_s=.20 were interpreted as weak, above r_s=.40 as moderate and above r_s=.60 as strong. r_s was used since the variables were found to be nonnormal distributed upon examination, and since the alliance ratings are based on ordinal data.

Analyses further assessed the relationship between psychological flexibility and quality of life as well as remission of diagnoses. These analyses can be found in Appendix II (Supplementary Material).

Correlations for patient- and therapist-rated alliance were assessed using r_s and by examining the relationship for i) expected alliance by using the scores for alliance estimates before treatment; ii) for alliance rated in the middle of treatment (week 3); iii) for development of alliance by evaluating changes in alliance- ratings between week 3 and week 7 in treatment, with the difference between pre- and post-measurement of anxiety estimates (assessed with SCAS-S). In addition, correlations for the 3 subscales of the alliance ratings (task, bond, goal) were examined. The same analyses as for the patient-rated alliance estimates were performed for the therapist-rated alliance.

Results

Pre-treatment testing

No statistically significant between-group differences were found for demographic variables (p=.10 to p=.72; Table 1), outcome measures (p=.16 to p=.78), or the number of fulfilled diagnoses (p=.65) at the pre-treatment assessment. No significant pre-treatment differences between completers and those lost to follow-up were observed for outcome measures (p=.16 to p=.82) or fulfilled diagnoses number (p=.612). However, there was a statistically significant difference in the demographic variable of age between completers and those lost for followup where those lost for follow-up had a higher mean age (M=17.30; SD=1.3; p=.048).

Effects of internet-delivered acceptance and commitment therapy on adolescents

Treatment adherence

Among the participants in the intervention group, 77% (n=21) completed more than half of the treatment program (4-8 modules), of whom 26% (n=7) completed all the treatment modules. One participant was never active in the treatment program. 17.4% stated that they had spent an average of 2 to 3 hours per week working with the program, 52.2% said they had spent 1 to 2 hours per week, and 26.1% had spent less than an hour per week.

Of the 52 included participants, 42 (23 from the intervention group and 19 from the control group) answered the posttreatment assessment questionnaires. Post-assessment diagnostic interviews were performed with 35 of the 45 clinical participants (20 from the intervention group and 15 from the control group). Table 3 shows descriptive statistics for measurement points for all the outcome variables.

Anxiety, quality of life, and psychological flexibility

The results from MLM for repeated measures showed that the iACT intervention group improved from pre- to post-assessment on each continuous outcome measure; SCAS-S (df=40,6; F=25,7; p<.001), GAD-7 (df=40,6; F=13,0; p<.001); BBQ (df=41,8; F=50,0; p=<.001) and AFQ-Y8 (df=40,0; F=32,4; p<.001). Effect sizes for observed values between pre- and postassessment showed a large with-in group effect sizes for the intervention group on SCAS-s of d_{RM, pool}=1.14, 95% CI [-1.78, .50], a moderate effect on GAD-7 of d_{RM, pool} =.71., 95% CI [−1.33, .10], a large effect on BBQ of d_{RM, pool} =3.15, 95% CI [2.47, 3.82] and a large effect on AFQ-Y8 of d_{RM, pool} =1.52, 95% CI [-2.18, - .86]. Results further showed significant interaction effects between the two conditions for the outcome measures of BBQ (df=41,8; F=6,9; p=.012) and AFQ-Y8 (df=40,0; F=5,73; p=.022), where the intervention group showed a significantly larger improvement in quality of life and psychological inflexibility than the control group. Effect sizes based on observed values showed moderate between-group effects for BBQ of d_S=.65 [-6.03, 8.42] and for AFQ-Y8 of d_S=−.51 [-3,19, 2.90]. However, no time×group interaction effect was observed in outcomes for anxiety, neither on SCAS-S (df=40,6; F=2,8; p=.103) or GAD-7 (df=40,6; F=1,5; p=.225). Effect sizes based on observed values showed small between-group differences of d_S=−.24 [-4.34, 3.48] for SCAS-s and of d_S=-.23 [-2.58, 2.43] for GAD-7.

Remission of diagnosis

Results from the analyses of the remission of initial diagnoses are based on a total of 35 clinical participants who answered the post-assessment diagnostic interview with MINI-KID (n=35; n treatment group=20; n control group=15). At post-assessment, 13 out of the 20 assessed participants in the experimental group (65%) had reached remission from their initial anxiety diagnose/diagnoses. Only one out of the 15 assessed participants in the control group (6.7%) had reached remission of their initial anxiety diagnose/diagnoses. Results from a chi-squared test showed a statistically significant difference in post-treatment diagnoses between the intervention and control groups [χ²(1)=12.15, p<.001].

Associations between psychological flexibility and outcomes for anxiety

All analyses of correlations with psychological flexibility are based on actual values and data from participants in the intervention group only. The results from the analyses using r_s showed a strong statistically significant relationship between changes in psychological flexibility and changes in symptoms of anxiety (assessed on SCAS-S) between the preand post-treatment measures for the intervention group (r_s=.50, p=.014, n=23). An equal relationship was observed when anxiety was assessed on GAD-7 (r_s=.68, p<.001, n=23). No significant association was found between psychological flexibility and quality of life or remission of diagnosis posttreatment.

Association between adolescents' perceived alliance and outcome

All 27 participants in the intervention group answered questions about the expected alliance before treatment. 20 participants (74%) answered WAI-SR at week 3, and 22 participants (81.5%) answered WAI-SR at week 7. Descriptive statistics for the alliance ratings are found in the Supplementary Material. Table 4 shows the results of the correlation analyses between patientrated alliance and outcomes for anxiety based on SCAS-S.

Participants’ expected alliance before treatment showed a trend towards significance and a close to a moderately strong relationship with outcome for anxiety (r_s=.39, p=.06, n=23). Subscale analyses showed significant and moderate relationships for expected Bond (r_s=.47, p=.02, n=22) and expected Task with outcome (r_s=.47, p=.03, n=23). Patient-rated alliance at week 3 showed a non-significant weak association with the outcome (r_s=.22, p=.35, n=20) as well as no significant association for the respective subscales. No significant association was found for alliance development during treatment and outcome (r_s=.10, p=.68, n=19).

Table 3.

Descriptive statistics for measurement points on Spence children’s anxiety scale-short version, generalized anxiety disorder 7- item, Brunnsviken brief quality of life scale and avoidance and fusion questionnaire for youth.

	Treatment condition
	Intervention group		Control group
	Pre-treatment (n=27)	Post-treatment (n=23)	Pre-treatment (n=25)	Post-treatment (n=19)
SCAS-S
Mean	28.6	20.5	25.8	22.6
SD	7.0	10.0	7.2	8.3
Range	16-39	8-40	14-44	10-41
GAD-7
Mean	12.3	8.6	11.0	9.9
SD	3.7	5.8	5.1	5.9
Range	6-19	1-21	3-21	0-20
AFQ-Y8
Mean	20.2	13.3	19.3	16.8
SD	5.2	6.6	5.5	7.6
Range	9-30	2-28	7-28	3-30
BBQ
Mean	36.5	55.4	37.8	44.8
SD	19.1	16.3	16.3	17.3
Range	8-96	28-88	6-61	14-76

SCAS-S, Spence children’s anxiety scale-short version; GAD-7, generalized anxiety disorder 7-item; AFQ-Y8, avoidance and fusion questionnaire for youth; BBQ, Brunnsviken brief quality of life scale; SD, standard deviation.

Association between therapist-rated alliance and outcome

The therapist alliance ratings had a response rate of 100%. Associations between therapist-rated alliance and outcome for anxiety based on SCAS-S were all weak and non-significant, except for the subscale task for the expected alliance which showed a medium significant association with outcome (r_s=.46; p=.03, n=23). Table 5 shows the results of the correlation analyses between therapist-rated alliance and outcome.

Treatment acceptability

Most participants (73.9%) stated that they would recommend the treatment to a friend with similar problems, 26.1% stated that they might recommend the treatment, and no participants said they would not recommend the treatment. Participants were also asked which interventions in the treatment program they had found most useful. It was possible to choose several options and the two interventions that most participants rated as helpful were functional analysis and cognitive defusion (52.2%). Next came information and individual mapping of anxiety (43.5%), mindfulness (30.4%), and exposure (26.1%).

Discussion

This RCT was aimed to assess the effectiveness of an iACT treatment for adolescents aged between 15 and 19 with anxiety disorders and recruited from all over Sweden. The study also aimed to assess key mechanisms of change in treatment, namely psychological flexibility and alliance, and their relationship to treatment outcomes for the main problem of anxiety. The iACT treatment was considered an acceptable intervention, with twothirds of the participants stating that they would recommend it to a friend with similar problems. The participants rated several interventions in the ACT treatment as helpful in dealing with their problems, favoring cognitive defusion and functional analysis most prominently, along with interventions such as exposure, mindfulness, and psychoeducation about symptoms.

The overall results showed that the treatment was effective in increasing participants' reported quality of life and psychological flexibility, with moderate between-group effect sizes (d_S=.65 and d_S=.51, respectively). Participants in the intervention group were also significantly more likely to no longer meet the criteria for any of their initial anxiety diagnoses after treatment (65%) than those in the control group (6.7%). These results are in line with prior research indicating that ICBT is an

Table 4.

Results of the correlation analysis between patient-rated alliance and outcome with Spence children’s anxiety scale-short version (difference between pre- and post-assessment).

	SCAS-S outcome
	rs	p
Expected alliance
Total scale (n=23)	.39	.06
Subscale bond (n=22)	.47*	.02
Subscale task (n=23)	.47*	.03
Subscale goal (n=23)	.18	.42
Alliance rated at week 3
Total scale (n=20)	.22	.35
Subscale bond (n=20)	.09	.70
Subscale task (n=20)	.14	.55
Subscale goal (n=20)	.26	.26
Development of alliance rated at weeks 3 and 7, total scale (n=19)	.10	.68

SCAS-S, Spence children’s anxiety scale-short version; rs, Spearman’s Rho coefficient. *p<.05.

Table 5.

Results of the correlation analysis between therapist-rated alliance and outcome with Spence children’s anxiety scale-short version (difference between pre- and post-assessment).

	SCAS-S outcome
	rs	p
Expected alliance
Total scale (n=23)	.27	.21
Subscale bond (n=23)	.26	.24
Subscale task (n=23)	.46*	.03
Subscale goal (n=23)	-.001	.996
Alliance rated at week 3
Total scale (n=23)	.09	.69
Subscale bond (n=23)	-.04	.86
Subscale task (n=23)	.05	.82
Subscale goal (n=23)	.10	.64
Development of alliance rated at weeks 3 and 7, total scale (n=23)	.24	.27

SCAS-S, Spence children’s anxiety scale-short version; rs, Spearman’s Rho coefficient. *p<.05.

However, the results showed no significant between-group difference over time for the participants' self-rated anxiety symptoms. Interestingly, this result is in line with a recent metaanalysis of ICBT for patients <18 years with pediatric anxiety, where ICBT showed effectiveness on remission of primary anxiety diagnosis and overall functioning as rated by clinicians, but not for anxiety-symptoms as assessed by self-report measures of participating youths (Cervin & Lundgren, 2022). This is a finding that could be further explored in future research. However, the results could also be seen in light of the theoretical foundation for ACT, which the evaluated treatment program was based on. The purpose of any ACT treatment is to help patients identify a meaningful and rich life that expresses their own values, commit to acting following these values, and learn to accept that suffering can arise and be part of this journey (Hayes et al., 2012). Therefore, symptom reduction is not the main focus of ACT. In fact, when patients act in accordance with what is important to them, their symptoms of anxiety or distress may increase because these behaviors may be anxietyprovoking for them. Thus, the goal of ACT is to help patients accept these feelings and not let them become a barrier to doing things they find meaningful. Although symptoms of anxiety remained after the treatment period, it is possible that they no longer posed the same difficulty for participants as they did before treatment. According to most psychiatric diagnostic interviews, including the one used in this study (Sheehan et al., 2010), symptoms per se are not a sufficient criterion for an anxiety diagnosis unless they also pose a functional impairment. Thus, this could explain why the participants in the treatment group showed a post-treatment difference in diagnoses, quality of life, and psychological flexibility, suggesting increased function, even though their symptoms of anxiety were not reduced to the same degree.

However, the study also faced the influence of the COVID- 19 pandemic, which occurred about 6 weeks into the treatment period. The pandemic’s circumstances could have impacted various aspects of the study results. When participants spent more time at home due to distance education their exposure to anxiety- inducing situations might have been reduced, temporarily relieving their anxiety. This could explain why the control group also experienced a decrease in anxiety symptoms. On the other hand, the pandemic may also have had an impact on the intervention participants’ abilities to execute certain parts of the treatment, such as exposing themselves to anxiety-provoking situations or doing things in line with their valued direction (e.g., gathering in larger groups or being physically close to people), which would be at odds with the Swedish Public Health Agency’s recommendations during the pandemic. In many traditional CBT and ACT protocols exposure is a key element in reducing anxiety and fear, but only 26% of participants considered it highly useful in this study. On the other hand, symptoms of depression and anxiety were generally found to increase in adolescents during the first year of the pandemic (Racine et al., 2021), contradicting the results of this study. However, symptom rates were higher in studies conducted in the latter part of the first year of the pandemic, suggesting that the effect of social isolation, school interruption, and concern for family members or family financial difficulties accumulated over time (Racine et al., 2021). This study was conducted early in the pandemic and participants had completed about half of the treatment before school restrictions were implemented. Therefore, it is uncertain to what extent the consequences of the pandemic affected the study's results.

Lastly, another consideration is that both the intervention group and the control group showed some improvements in their self-rated anxiety symptoms on the primary outcome measure SCAS-S between the pre- and post-assessments. However, through the interaction term in the statistical model you want to know if there is a substantial improvement that could be attributed to the treatment intervention alone. Many mental health problems have some degree of natural recovery that takes place regardless of whether treatment is started, and the fact that also the control group showed improvements in their anxiety symptoms could possibly be attributed to spontaneous recovery. Yet another explanation could be attributed to measurement reactivity, meaning that people might start to behave differently, or symptoms might diminish, by the sole fact that participants know they are being measured. Considerations should also be made about statistical power since this study was a relatively small study including 52 participants. The absence of a significant interaction effect might be the result of limited power for finding statistical effects.

This study also explored key mechanisms of change in the treatment and hypothesized psychological flexibility and alliance as important processes for successful treatment outcomes. Other than knowing that treatments work, it is of relevance to investigate which mechanisms in treatment bring about change. Results showed a strong statistically significant correlation between changes in psychological flexibility and changes in anxiety symptoms for the intervention group as assessed on both SCAS-S and GAD-7. Higher psychological flexibility was related to greater improvement in symptoms, or conversely, greater anxiety improvement may have corresponded to higher psychological flexibility. Since the goal of any ACT treatment is to improve psychological flexibility, this indicates some support for the ACT treatment as an own active ingredient in the treatment outcome as opposed to just spontaneous recovery or expectation effects. This is in line with previous research that has shown psychological flexibility as an important mechanism of change in treatment outcomes (Stabbe et al., 2019). However, no formal mediating analysis of psychological flexibility was performed in this study; therefore, it is not possible to make any causal claims (Cuijpers et al., 2019). A mediational analysis would be preferable to assess psychological flexibility during treatment, capture when change occurs, and follow up to explore change over time to reach any conclusions on psychological flexibility as a working mechanism (Hayes et al., 2022). Moreover, in this study, psychological flexibility was captured with the self-assessment scale of AFQ-Y8, which is broadly used as an ACT process measure in research. However, the AFQ-Y8 measures only parts of the 6- folded model of psychological flexibility, namely cognitive defusion and acceptance (Greco et al., 2008). Currently, not much research exists on valid and reliable measurements that captivate all the six core processes of the model for psychological flexibility. Such a measure would be preferable in future mediation studies of internet-delivered ACT for adolescents.

Regarding the other key mechanism explored in this study, the alliance was in general rated as high by both the participating adolescents and the therapists, which indicates that it is possible to create a positive alliance in internet-delivered treatments for adolescents. This is in line with prior studies on alliance in internet-delivered treatments for adults (Andersson et al., 2012). In a newly published study on therapists’ experiences of treating adolescents in ICBT, the therapists reported that the written messages by the adolescents were in some cases richer than in standard face-to-face CBT (Weineland et al., 2020). They described a category of patients for whom self-disclosure in the therapy room was difficult, but who were more open to sharing shameful feelings and thoughts through writing. This is consistent with the assumption that it is possible to create a positive bond in an internet-delivered treatment format. However, no significant associations were found between either patient-rated or therapist-rated alliance and treatment outcomes for anxiety in this study. This contrasts with the predictive value of a positive alliance in face-to-face treatments (Flückiger et al., 2018; Karver et al., 2018), but aligns with prior research on alliance in ICBT for adults. There is usually no correlation, even though the alliance estimates are generally high in ICBT treatments (Andersson et al., 2019). Although the alliance is of great importance for the outcome of psychological treatment in faceto- face treatments, it is not obvious whether it is equally important in internet-delivered treatments. The communication between the therapist and the patient is mostly in written form, and the therapist and patient may not know each other face-toface. Some researchers have claimed that the patient rather creates an alliance with the material in the treatment (Andersson et al., 2012), where the patient through texts, films, and sound clips takes part in a narrative based on one or more main characters and/or a virtual therapist. Therefore, traditional assessment scales for capturing alliance, including the measurement scale used in this study (Munder et al., 2010), may not be equally valid in an internet-delivered treatment and more refined measurement instruments might be needed to evaluate whether the alliance has any predictive relationship in ICBT (Andersson et al., 2012). Apart from this, the study results showed a trend towards significance (p=.06) and a nearly moderate strength association between patient-rated expected alliance before treatment and outcomes for anxiety. Additionally, subscale analyses showed that patient-expected bond and task had a significant and moderate relationship with outcome. When the alliance was rated by therapists, the expected task was associated with treatment outcomes. Hence, future studies could explore whether it is important to create high expectations in the patient before treatment and whether motivational aspects of the task of treatment are of special importance.

Limitations and directions for future research

This study included participants in a convenience sample recruited through advertisements on social media, and in schools, primary healthcare centers, and outpatient psychiatric clinics. However, the treatment has not been tested in a clinical setting, and the participant group in this study was self-selected, which might limit the generalizability of the study results. Based on the severity of the participants’ symptoms, as shown by their results on the main outcome measure SCAS-S before starting treatment, and the number of diagnostic criteria they met before treatment, it can be argued that participants in this study constituted a clinical group. Many had also received previous psychological treatment which strengthens the notion that the participants were adolescents who would have been found within a healthcare context had they not participated in the study. However, future research may benefit from examining the effect and implementation of such a treatment in a clinical context.

Some additional considerations could be made in terms of the study sample representativeness. At the pre-assessment for this study, many participants fulfilled the criteria for more than one anxiety diagnosis, and the prevalence of agoraphobia in this sample was higher than what would be expected within the general population. This raises questions about the validity of the structured diagnostic interview based on MINI-KID and whether it can accurately distinguish between psychiatric disorders in a sample of adolescents. This is interesting in light of some evidence that suggests the agreement on disorders when administered to both adults and youths is low (Duncan et al., 2019). On the other hand, one could also speculate that patients with panic disorder and/or agoraphobia seek care more often than patients with other anxiety diagnoses and that, therefore, these patients may be overrepresented within healthcare as well as in treatment studies. Moreover, in this study, some participants (n=7) with subclinical anxiety were included, which possibly could cloud the results. However, the subclinical participants had signed up for the research study based on significant suffering and were seeking help, thus a decision was made to include them. The subclinical participants did not differ significantly on demographic variables, quality of life, or initial anxiety severity as measured by SCAS-S compared to the clinical participants; however, they presented with higher psychological flexibility and lower scores on the secondary measurement for anxiety, GAD-7, at pre-assessment. Finally, internet-based interventions have been proposed as a means for reducing health disparities and differences in access to healthcare that exist between different socio-economic, ethnic, and language groups (Ralston et al., 2019). Demographic variables in terms of ethnicity and socio-economic status were not collected for this study; however, the inclusion criteria included being able to speak and understand Swedish adequately. Thus, this might have affected the study sample’s representativeness. Moreover, around 80% of the participants in this study were women. However, the last-mentioned problem could be argued to align with the clinical practice, since a considerable majority of adolescents that seek help for mental health problems within healthcare are women.

Participants in this study received weekly written feedback as well as telephone support on 3 occasions: before the start, in the middle, and at the end of the treatment. The use of the secure messaging function in the program varied between participants: some participants seldom replied to their therapist and others both replied and contacted the therapist. No data exist on how motivational contact varied by participants for this study, and this could be explored further since it might be another factor that could influence the working alliance.

This study was designed as an RCT comparing an intervention group receiving iACT to a wait-listed control group. Therefore, it is not possible to fully rule out expectation and demand characteristics as possible explanations for the differences found between the treatment conditions. Moreover, the effect of the intervention was evaluated only by post-treatment with no further follow-up. Challenges for digital mental health studies include evaluating treatments against adequate control conditions, evaluating their effectiveness in routine clinical care, and identifying and resolving potential barriers to the implementation of these interventions in the real world, such as education and training for clinicians and ensuring the interventions are appealing and accessible to patients (Katharine et al., 2023). Therefore, future studies may benefit from examining the efficacy of iACT for youth in comparison to an active control group, conducted in routine clinical care and with long-term follow-ups of treatment effects.

Conclusions

This study shows promising results for iACT treatment for adolescents with anxiety disorders. The results showed that the treatment was effective in increasing the adolescents’ perceived quality of life and psychological flexibility, and it had a positive effect on post-treatment diagnoses in the intervention group. A strong association was found between changes in psychological flexibility and changes in anxiety symptoms, where higher psychological flexibility was related to a greater improvement in symptoms. This indicates that psychological flexibility might be an important process to support during treatment. However, no significant between-group difference over time was seen for self-rated anxiety symptoms. Measures of working alliance indicated that an alliance can be successfully established in iACT for adolescents; nevertheless, the alliance had no significant relationship with the outcome in this study. The participating adolescents perceived the treatment as an acceptable intervention. Future research should validate the findings from this study in larger samples, in clinical settings, and using formal mediational analysis.

Funding Statement

Funding: this study was sponsored in part by R&D Primary Health Care, Västra Götaland region. R&D Primary Health Care had no role in the design of the study, the data collection or analysis, the writing of the report, or the decision to submit the article for publication. Availability of data and materials: the raw data that support the findings of this study are available from the corresponding author, LN, upon request.