Skip to main content
PMC home page
Internet Interventions logoLink to Internet Interventions
. 2018 Nov 2;15:10–17. doi: 10.1016/j.invent.2018.10.001

The role of knowledge in internet-based cognitive behavioural therapy for adolescent depression: Results from a randomised controlled study

Matilda Berg a,⁎, Alexander Rozental b,c, Sofie Johansson a, Lina Liljethörn a, Ella Radvogin a, Naira Topooco a, Gerhard Andersson a,b
PMCID: PMC6260280  PMID: 30519531

Abstract

Background and objectives

Clients' knowledge about their condition and treatment is considered crucial for general health improvement, and knowledge acquisition is an essential part in internet-based cognitive behavioural therapy (ICBT). Yet, little is known about the role of knowledge and how it influences treatment outcome. This study aimed to examine if explicit knowledge increased following ICBT for adolescent depression, if knowledge gain would be associated with symptom reduction, and if pre-existing knowledge predicted changes in depressive symptoms.

Methods

Seventy-one adolescents were randomised to a therapist-supported ICBT or a attention control condition. A measure of depression (BDI-II) and a knowledge test dealing with depression, comorbid anxiety, and its CBT-treatment were administered before and after treatment.

Results

Significant improvements in knowledge were observed following ICBT compared to the attention control (between-group Cohen's d = 1.25, 95% CI [0.67–1.79]). On average, participants in the treatment group answered 1.4 more questions correctly at post treatment compared to the control group. No relation between change in knowledge and change in depressive symptoms could be observed. Knowledge scores at baseline were high for both groups, with participants answering approximately 75% of the questions correct. A higher level of initial knowledge level predicted poorer treatment response (Parson's r = −0.38, p = .048).

Conclusions

The findings indicate that knowledge about basic concepts and principles about depression, anxiety, and CBT increases following ICBT. This increase in knowledge was not related to change in depressive symptoms, indicating that knowledge is a different construct. The results also suggest that clients who are more knowledgeable prior to treatment might benefit less from ICBT. In sum, the results highlight the need to further examine the role of knowledge in ICBT.

Keywords: Internet-based cognitive behavioural therapy, Knowledge, Adolescent depression

Highlights

  • •

    The role of explicit knowledge was examined in an ICBT treatment for adolescent depression.

  • •

    Explicit knowledge increased following treatment.

  • •

    Initial knowledge might be an important predictor for treatment success.

  • •

    The role of explicit knowledge needs to be examined further in ICBT.

1. Introduction

One neglected topic in research, but a core feature in psychotherapy, is the role of knowledge and learning (Harvey et al., 2014). This is a field of particular relevance in the treatment of children and adolescents as the first onset of depression in adolescence is often followed by later depressive episodes (Thapar et al., 2012), making knowledge important to explore because of its possible protective effects. Knowledge and psychoeducation have also been highlighted as important factors in many areas of health care, based on the assumption that more well-informed consumers of health care will have better treatment outcomes (Lukens and McFarlande, 2004). For example, psychoeducational interventions (an inherent part of cognitive behavioural therapy; CBT) have been found to improve quality of life and individual functioning, as well as decrease psychological symptoms in clients with cancer and schizophrenia (Lukens and McFarlande, 2004). Moreover, in combination with medication, psychoeducation can increase adherence to treatment in patients with bipolar disorder (Gonzalez-Pinto et al., 2004), and improve psychosocial function, adherence to treatment, and decrease symptoms in depression (Tursi et al., 2013). Psychoeducation aims to aid and empower patients to take a more active role in managing their problems, improving their cognitive mastery and coping strategies, which in turn is thought to result in better health outcomes and prevent relapse (Ryhänen et al., 2010; Sajatovic et al., 2007). However, most of this research has dealt with adults and not adolescents or children.

Despite the potential importance of knowledge and learning in health-care, few studies in the field of psychotherapy have evaluated what clients actually learn from their therapy and if knowledge acquisition is important for outcome. This includes research with younger clients. One purpose of CBT is to involve clients in more adaptive learning experiences, targeting both declarative (explicit) knowledge through verbally accessible interventions, such as psychoeducation or self-observing strategies, as well as nondeclarative (implicit) knowledge gained through exercises such as behavioural activation and exposure (Brewin, 1996). Given the emphasis on psychoeducation and providing a rationale for the interventions used in treatment, evaluating the role of explicit knowledge acquisition is arguably important in CBT, and in particular in internet-based cognitive behavioural therapy (ICBT) (Andersson et al., 2012; Harvey et al., 2014). Both CBT and ICBT emphasise educational components that intend to improve clients' knowledge of their condition in order to enable them to change their situation based on their new insights (Friedberg et al., 1998; Scogin et al., 1998; Andersson et al., 2012). There is also emerging research highlighting the potential importance of targeting and enhancing explicit learning and memory of treatment content in order to improve treatment outcome in face-to-face CBT (Harvey et al., 2014; Dong et al., 2017). ICBT, on the other hand, is often based on texts (Andersson, 2016), and relies on generic educational components from CBT. Clients are assumed to benefit from the written material by comprehending the information and use it to change their behaviour and master new skills. Without as much interaction with clinicians as in traditional CBT, it is difficult to gauge what clients learn during treatment, which makes knowledge acquisition particularly important to evaluate in the context of ICBT. In the case of treatment of adolescents this can be even more important as most ICBT studies on adolescents involve written instructions (Ebert et al., 2015), and language comprehension may vary substantially.

ICBT can be regarded as an ideal context to study the role of knowledge and learning (Harvey et al., 2014), and adolescents could be a suitable target group for such research. Moreover, studies on the efficacy of ICBT could benefit from evaluating clients' explicit knowledge and its relation to treatment outcome.

Several studies have highlighted the importance of knowledge via evaluations of psychoeducational interventions (Lukens and McFarlande, 2004; Tursi et al., 2013). However, many of these studies have not measured knowledge levels before and/or after treatment, and researchers in this field have also stressed the importance of using knowledge as an outcome measure (Lukens and McFarlande, 2004). Several trials show that knowledge increases following internet education for a range of disorders (Fox, 2009; Ryhänen et al., 2010), but the relation between knowledge increase and intervention outcome has rarely been reported.

While there are intervention studies in which knowledge has been measured, most studies lack descriptions of test development procedures and reports of the psychometric properties of the measure used. Overall, the field suffers from several methodological challenges. For instance, when evaluating mental health literacy, knowledge gain is mostly measured through vignettes, with vast differences in type of scoring-system and formats used (O'Connor et al., 2014). Overall, a challenge when constructing knowledge tests is to develop generic tests that cover all important aspects of a construct. When choosing and defining which area of knowledge to assess, there is a range of different alternatives such as coping skills, treatment content, or theories behind the interventions.

There are a few studies on CBT and ICBT for adults in which knowledge has been evaluated (e.g., Andersson et al., 2012; Friedberg et al., 1998; Christensen et al., 2004; Griffiths et al., 2004; Kronmüller et al., 2007; Scogin et al., 1998; Strandskov et al., 2018). For instance, Andersson et al. (2012) randomised a total of 204 participants with social anxiety disorder (SAD) to guided ICBT or a control group, and found that basic knowledge about SAD and its treatment increased following treatment compared to the control group. While the study showed knowledge gains in the treatment group, there was no relation between knowledge and the primary outcome measure on social anxiety, but a small and significant correlation between knowledge gain and secondary outcome on SAD. Friedberg et al. (1998) evaluated the role of psychoeducational knowledge by testing 123 depressed clients in a Cognitive Therapy Unit, showing that general knowledge about CBT principles improved during treatment. They did not find any association between knowledge and outcome on depression. Christensen et al. (2004) randomised 525 participants with depression to either an information website, a cognitive-behavioural skills training website, or a control condition. By measuring depression literacy using a test about depression and the principles in CBT they found that both websites improved knowledge about effective treatments. The educational website lead to improvement in knowledge about treatments in general and the cognitive-behavioural website about CBT in particular. The correlation between health literacy and depressive symptoms was however not reported. In a prospective study, Kronmüller et al. (2007) evaluated the predictive value of general knowledge by testing 62 clients with major depression on a test measuring knowledge about depression and mania. The authors found that general knowledge about treatment, but not general knowledge about coping, was predictive of depressive symptoms two years later.

Taken together, studies suggest that knowledge increases during treatment. However, the relation between knowledge and symptom reduction and treatment outcome remains unclear. None of the studies have evaluated the effect of initial knowledge on treatment outcome. Prior knowledge can have a marked effect on learning outcomes in the context of educational research (Shapiro, 2004). Further it is important to identify clinically relevant predictors of outcome in ICBT, since the literature on outcome predictors of ICBT is inconsistent and scattered (Andersson, 2016). Even less is known regarding knowledge in ICBT for adolescents.

It is however not just actual knowledge per see that is of interest in the present study, but also how certain clients are with regard to their knowledge. The distinction between certainty and accuracy is commonly measured in cognitive psychology (Tiedens and Linton, 2001; Moore and Healy, 2008). The need to assess certainty when measuring knowledge in therapy has been stressed previously from a clinical point of view, as certainty could aid clients to apply what they have learned during treatment (Friedberg et al., 1998). To our knowledge, incorporating certainty ratings when measuring knowledge in ICBT has only been implemented in two previous studies with adult samples (Andersson et al., 2012; Strandskov et al., 2017). Andersson et al. (2012) constructed a test with eleven multiple choice items related to SAD and its treatment, and incorporated certainty ratings using a 3-point scale. Strandskov et al. (2018) developed a multiple-choice test with 16 items to measure knowledge about eating disorders and its treatment. They included ratings of certainty for the knowledge items using a 4-point scale. Incorporating certainty ratings is a way to evaluate whether clients actually have gained the correct knowledge rather than guessing their answer. Thus, outcome of a knowledge test can either be scored in terms of raw scores (correct answers) or as a weighted score with certainty ratings being incorporated (Andersson et al., 2012).

This study is a secondary report of a previously published randomised controlled trial on ICBT for adolescents in which significant treatments effects were reported for psychiatric outcomes (Topooco et al., 2018). In the present report, we focus on the role of explicit, declarative knowledge in the context of adolescent depression (not reported in the original trial). To our knowledge, no study has explored the importance of knowledge in relation to outcome in ICBT for younger clients. The main aim of the study was to investigate changes in explicit, declarative knowledge about ICBT and depression by constructing a brief self-report measure of knowledge and to use the measure before and after ICBT for adolescent depression. Both raw knowledge scores and weighted scores based on certainty ratings were assessed. We also investigated if acquired knowledge would be associated with treatment outcome. Finally, we examined if pre-treatment knowledge could predict changes in depressive symptoms after treatment.

2. Method

2.1. Recruitment and selection procedure

As mentioned, this study was a part of a randomised clinical trial in which the main symptom outcomes were presented, and in which more details about the trial are presented (Topooco et al., 2018). Here, we focus on the role of knowledge in relation to the outcome of depressive symptoms. Recruitment and assessment took place between January and February 2015. The recruitment procedure involved advertisements through mass media in Sweden. Participants were primarily recruited through 50 high schools in Sweden. In addition, several relevant care professionals in Sweden (e.g., psychologists, counselors, and nurses), and Swedish organisations for youth, helped to spread information in relevant public areas. Potential clients could register their interest to participate in the study via a research platform presenting the project (Vlaescu et al., 2016). This platform presented information about the treatment, the staff, eligibility criteria, informed consent and provided an initial online screening including all outcome measures. Clients who passed the initial screening were contacted for a diagnostic interview by phone, using the Mini International Psychiatric Interview (M.I.N.I.; Sheehan et al., 1998). The assessors using the M.I.N.I. were trained in how to administer the interview before the assessment started. They received supervision throughout the whole recruitment process. The M.I.N.I. was also administered at post treatment, but at that point only those questions related to the participants' diagnoses at baseline were used in the interview.

Eligible criteria were that participants should 1) be between 15 and 19 years old; 2) meet the criteria for major depression or presenting with a minimum of five symptoms of the disorder, according to Diagnostic and statistical manual of mental disorders (DSM-V-TR; American Psychiatric Association, 2000) using the M.I.N.I., and/or score ≥ 14 on the Beck Depression Inventory-II (BDI-II; Steer et al., 1998); 3) have no or low risk of suicide according to the M.I.N.I. and the Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001); 4) not suffering from any co-morbid conditions interfering with the treatment, according to the M.I.N.I.; 5) no ongoing psychological treatment that could affect the ICBT-treatment; 6) if medicated for depression or anxiety, the medication had to be initiated at least three months prior to treatment, with one month of stable dosage; and 7) not have too severe problems (e.g. bipolar disorder or schizophrenia), nor 8) abuse drugs or alcohol according to the MINI and the Alcohol Use Disorders Identification Test (AUDIT; Saunders et al., 1993). For AUDIT, we used the cut-offs ≥8 points for men, ≥6 points for women as guidelines. Furthermore, participants had to master Swedish in both written and spoken language.

The selection procedure was based on the initial screening, the telephone interview, and clinical judgements during referral conferences. Those not eligible received a personal explanation as to why they were not included and advice on how to seek other health care services. Included clients received treatment through a secure platform requiring two-step authorization (Vlaescu et al., 2016). The treatment was delivered between February and April 2015. The control group received treatment immediately after the treatment group (see Topooco et al., 2018).

This trial was approved by the regional ethics committee in Linköping, Sweden (Dnr 2014/427-31), and was registered at ClinicalTrials.gov (ID: NCT02363205). Informed consent was obtained from all participants.

2.2. Randomisation procedure

Seventy-one individuals of the initial 139 potential clients who completed the screening process were randomised in a 1:1 ratio to ICBT (n = 34) or control (n = 37). The randomisation procedure was performed through www.random.org by a person not involved in the study. One included client in the treatment condition declined to participate just before treatment started and was thus excluded.

2.3. Participants

A majority of the participants (total N = 70) were girls (94%) and attended high-school (94%). Their mean age was 17.2 years (SD = 1.0). A majority lived with both of their parents (73%). See Table 1 for further details. According to the BDI-II, 89% of the total sample had moderate to severe depression (score > 20). In the treatment group about 85% (n = 28) met criteria for Major Depressive Episode and 15% (n = 5) suffered from Depression with core symptoms. In the control condition 68% (n = 25) met criteria of Major Depressive Episode and 32% (n = 12) suffered from Depression with core symptoms. For more details regarding the sample see Topooco et al. (2018).

Table 1.

Baseline characteristics and symptoms of participants.

Variable Treatment
n = 33		 Control
n = 37
Age, m (sd) 17.2 (1.0) 16.9 (1.1)
Girls, n (%) 31 (93.9) 35 (94.6)
Knowledge score m (sd)
 Raw scores 13.20 (2.5) 13.03 (2.0)
 Weighted scores 17.27 (4.7) 16.54 (3.6)
BDI-II score
 Group mean m (sd) 33.1 (9.4) 32.3 (10.2)
Duration of depressive symptoms n (%)
 0–6 months 2 (6.1) 5 (13.5)
 0.5 to 1 year 2 (6.1) 3 (8.1)
 1 to 2 years 11 (33.3) 5 (13.5)
 >2 years 18 (54.5) 24 (64.9)
Residence
 City 10 (30.3) 12 (32.4)
 Small town 14 (42.4) 15 (40.5)
 Non-urban 9 (27.3) 10 (27.0)
Previous mental health contact, n (%) 21 (63.6) 24 (64.7)
Previous treatment, n (%) 14 (27.3) 13 (35.1)
 Counsellor contact 6 (18.2) 5 (13.5)
 Psychotherapy 11 (33.3) 7 (18.9)
 Psychotropic medication 3 (9.1) 1 (2.7)
Open in a new tab

CBT = cognitive behavioural therapy, BDI = Beck Depression Inventory.

2.4. Outcome measures

2.4.1. Knowledge test

We constructed a knowledge test to measure declarative knowledge about depression, comorbid anxiety, and principles presented in CBT (for example the role of being active, how to handle negative thoughts etc.). The test items are presented in Appendix 1. The test aimed to capture declarative knowledge about basic concept, points, and ideas from the treatment modules. The test construction procedure was inspired by previously developed knowledge tests (Andersson et al., 2012; Kronmüller et al., 2007; Scogin et al., 1998), and guidelines on how to construct valid tests (Haladyna, 1994; Scogin et al., 1998).

The generation and selection of test items was related to the content of the ICBT-treatment. We also consulted clinical experts in the field of depression and anxiety, and discussed the item content in order to assure relevance in relation to CBT for depression. Further evaluation of the items involved pilot testing with psychology students and adolescents in the same age range as the target group of the study (between 15 and 19 years old) (n = 20). Some revisions had to be made to make the items more readable, comprehensible, and at an adequate level of difficulty. Approximately half of the items were specifically related to the treatment content, whereas other items were more broadly mentioned in the modules and dealt more with general aspect of depression, anxiety, and ICBT (see Appendix 1 for examples).

After revisions based on feedback from the experts, students, and adolescents the final test included 17 items. The first seven items were formulated as statements and had a true or false response option (inspired by Scogin et al., 1998). The ten following items had a multiple-choice response option (three response options) as in the tests described by Andersson et al. (2012) and Haladyna (1994). In addition, each response was rated in terms of experienced certainty (“Guessing”, “Pretty certain”, “Totally certain”).

The knowledge level was analysed in two ways. First, we calculated one raw score based on the total number of correct answers and then a weighted score in which the level of certainty was included in the calculation. If the participant was correct and certain the score was entered as positive (+1). If he or she was correct but uncertain a score of 0 was given, and the same if the person was incorrect but uncertain. If the participant was incorrect and certain a negative score (−1) was entered. Thus, higher scores on the test indicated more knowledge and certainty, while lower scores indicated less knowledge, including certainty about the wrong answer or being uncertain about the right answer. Participants could score between 0 and 17 points on the raw score calculations, and between −17 and 34 points on the weighted score. Reliability analyses showed an internal consistency (Cronbach's α) of 0.56 for the raw scores and 0.64 for the weighted scores.

2.4.2. Depression

To measure symptoms of depression, we used the Beck Depression Inventory-II (BDI-II; Steer et al., 1998), and this was the primary outcome in the controlled trial (Topooco et al., 2018). BDI-II can be used from ≥13 years and is the most common and validated way to measure adolescent depression (Zuckerbrot et al., 2007). For further description of other secondary outcome measures see Topooco et al. (2018).

2.5. Treatment

The treatment consisted of an 8-week guided Internet program, including psychoeducational modules with accompanying exercises (see more details in Topooco et al., 2018). All modules were based on established CBT-principles and previous studies on ICBT for adults (Johansson et al., 2012) and adolescents (Silfvernagel et al., 2015). The language and content of the original modules aimed for adults (e.g., Johansson et al., 2012) was adapted for the age group. The first module included an introduction to ICBT and depression. It introduced ICBT and how the treatment would proceed. Symptoms and causes of depression from a CBT-perspective were presented and explained. The second module focused on a functional analysis and how to analyse situations that lead to desirable and undesirable consequences. The third and fourth modules focused on behavioural activation, educating the clients about the link between behaviour and feelings. This included filling out a mood-activity diary to identify patterns of behaviour during the first treatment week and incorporating more meaningful activities during the second week. The fifth module focused on cognitive restructuring, explaining the relationship between thoughts and feelings and how to identify and challenge negative thoughts. The sixth week focused on comorbid anxiety, where general psychoeducation about anxiety and strategies on how to reduce it was presented (i.e. exposure). Here, the participants could also choose to tailor their treatment and read more specifically about certain forms of anxiety such as social anxiety or generalized anxiety. The seventh module included psychoeducation about affect regulation and self-esteem, focusing on how to cope with negative emotions and experiences about one's self-worth. The final module included treatment summary and psychoeducation about relapse prevention.

In between the sessions, the therapists provided written guidance and feedback within 24 h through the platform once completed homework assignments were received. In addition, each week the clients could meet their therapist for a brief scheduled chat-session online. Chat-conversations focused on answering questions about the treatment and the exercises. The therapists were final-year psychology students who received supervision on a weekly basis. The supervisors were clinical psychologists with extensive experience in treating adolescents with psychiatric problems. A total of 6 therapists was involved and they had approximately seven clients each (Topooco et al., 2018).

All the participants, including the attention control group described below, were instructed to fill in the PHQ-9 on a weekly basis through the platform. The PHQ-9 was primarily used for monitoring eventual increases in symptom severity. The therapists immediately contacted clients in the event of deterioration, as detected in the conversation and/or an increased score on the PHQ-9.

2.6. Attention control

Participants in the attention control were assigned to a therapist (from the therapists who also supported the ICBT group), and got restricted access to the platform where they could receive non-specific counselling (i.e. without the usage of specific CBT-techniques) and make contact or be contacted if their symptoms levels increased. A total of ten participants in the control group contacted their therapist due to symptom deterioration. None of the participants in the control group had to be excluded from the study and referred to other health care services (Topooco et al., 2018).

2.7. Statistical analyses

Statistical analyses were performed using SPSS version 24. Before the original trial we calculated the power to determine a sufficient sample size for the study (Topooco et al., 2018). This was not possible for the knowledge measure, however, since we lacked similar comparisons. For the BDI-II, we used an effect-size of d = 0.70 in line with similar studies on adults (Andersson et al., 2013), and a systematic review of ICBT for adolescents (Ebert et al., 2015). With an effect size of Cohen's d = 0.70, a power of 80% and an alpha of 0.5, the required sample size would be a total of 72 participants, 36 clients in each group. Hence, the final sample size (N = 70) in the trial was somewhat smaller but considered large enough to detect treatment effects for depression.

We used independent samples t-tests and χ2 test to detect possible baseline differences between treatment and control condition on continuous outcome measures and demographic/descriptive variables. In order to evaluate the differences between the treatment and the control group in knowledge level we used t-test. Levene's test was used to investigate whether the variance was homogeneous in the two groups.

Further, we applied analysis of covariance (ANCOVA) to analyse the difference between the two groups in knowledge level at post treatment, using baseline values as covariates (Vickers and Altman, 2001). We calculated treatments effects for the raw scores and also for the weighted scores.

In addition, we used Pearson's correlation coefficient to evaluate correlations between change scores in knowledge and change scores in BDI-II, and to examine the correlation between knowledge at pre-treatment and change scores on the BDI-II. In addition, we did additional analyses to control for potential confounding variables. We tested if severity of depression, number of completed modules, and earlier experience of treatment would affect the relationship between pre-existing knowledge and change scores on BDI-II.

Within-group effect sizes were assessed using the difference in means between the two measurement points for each condition, divided by the pooled standard deviation, while the between-group effect sizes (d) utilized the difference in means between the conditions at post-treatment assessment.

2.8. Missing data

Of the total sample (N = 70), seven (10%) did not complete the knowledge test at post treatment and four (6%) did not complete BDI-II at post treatment. Of the 65 participants who responded to the knowledge test at pre-treatment, 59 (91%) responded at post-treatment, and 66 of the 70 (94%) participants who completed BDI-II at pre-treatment answered BDI-II at post-treatment. No participant dropped out during the treatment period, i.e. the treatment drop-out was fairly low.

Given the fairly low percent of missing data post treatment we calculated the results according to the Complete Case analysis principle (Salim et al., 2008).

3. Results

3.1. Randomisation check

There were no statistically significant differences between the control group and the treatment group at baseline descriptive characteristics or continuous variables (t(68) = 0.273–0.313, p > .05; χ2 (1, 4) = 0.014–1.308, p > .05), including the knowledge test (t(63) = 0.301–0.697, p > .05).

3.2. Adherence

The treatment group completed an average of 6.48 of the eight mandatory modules (81%) (SD = 2.43). The average number of completed chat-sessions was 6.24 out of nine possible (69%) (SD = 2.60). In total, 23 of the 33 participants (70%) in the treatment group completed the whole treatment by completing all modules and attending all chat-sessions. On average, 91% of all participants, including the control group, responded on the weekly measurement of depression, using PHQ-9 (Topooco et al., 2018).

3.3. Knowledge acquisition

Data for both simple and weighted scoring on the knowledge test was checked for skewness and kurtosis. The analyses verified the assumption of normal distribution. No client had a negative knowledge score at post treatment and no client provided the wrong answer on an item with high certainty. Of the 26 clients who answered the knowledge test in the treatment group, two (7.7%) scored lower on knowledge (weighted scores) at post-treatment than prior to treatment. The two clients became less certain about the right answer during treatment. They did not differ however, in their change of depressive symptoms or on depression scores prior treatment compared to the 24 clients who improved their knowledge scores.

The descriptive statistics and results from the analyses evaluating knowledge and its relation to main depression outcome (BDI-II) are displayed in Table 2. The results from the ANCOVAs evaluating knowledge differences at post treatment showed a significant between-group effect with regards to knowledge levels in favour of the treatment group, both for the raw scores (F (1.56) = 5.09, p = .028 and for the weighted scores (F (1.56) = 13.44, p = .001). The effect size d was medium (0.67, 95% CI [0.14, 1.20]) for the raw scores, and large (1.25, 95% CI [0.67–1.79]) for the weighted scores.

Table 2.

Knowledge and depression scores of participants.

Instrument N Treatment
 Control
 Between-group pre- to post
 Within-group correlations with BDI change score
M (SD) N M (SD) F (df), effect size, Cohen's d (95% CI) Pearson's r
Knowledge test (raw scores)
 Pre 30 13.20 (2.59) 35 13.03 (1.99) 5.09 (1,56) −0.29
 Post 28 14.39 (2.28) 35 13.43 (1.82) 0.67 [0.14, 1.20]*
 Change scorea 26 1.42 (1.88) −0.03
Knowledge test (weighted scores)
 Pre 30 17.27 (4.73) 35 16.54 (3.62) 13.44 (1,56) −0.38*
 Post 28 21.50 (6.14) 35 17.66 (3.68) 1.25 [0.67, 1.79]*** −0.14
 Change scorea 26 4.73 (4.07)
BDI-II
 Pre 33 33.1 (9.4) 37 32.3 (10.2) 6.18 (1,67)
 Post 30 19.9 (7.2) 36 25.2 (7.8) 0.71 [0.22, 1.19]*
 Change scorea 30 12.8 (11.2)
Open in a new tab

Model derived means differences (SD) *significant p < .05, **significant p < .01, ***significant p < .001. Analysis adjusted for baseline score and based on Complete Cases analysis.

BDI-II = Beck Depression Inventory-II.

a

Change value relating to the difference between pre- and post-measurement.

As presented in the original report (Topooco et al., 2018), analyses on the primary outcome measure of depression (BDI-II) showed that the groups differed significantly (F(1,67) = 6.18, p < .05), with medium effect, d = 0.71, 95% CI [0.22–1.19]. Thus, the treatment had an effect on depressive symptoms.

3.4. Knowledge acquisition and depression

Results showed a non-significant correlation between change scores in knowledge and change scores on the BDI-II for both the raw and the weighted knowledge scores (r = −0.03, p = .875; r = 0.14, p = .502), see Table 2. The BDI-II scores at post treatment did not correlate significantly with the raw knowledge scores at post treatment (r = 0.36; p = .058), while weighted-knowledge scores did correlate significantly with BDI-II scores post treatment (r = 0.40; p = .033).

We also did a similar analysis on the PHQ-9 in order to further evaluate the relationship between knowledge and depression. This revealed a non-significant association between change in knowledge scores and change in depression (r = 0.19; p = .349; r = 0.05; p = .821). These results suggest that clients' improvement in knowledge and confidence in knowledge was independent of changes in depressive symptoms and vice versa.

3.5. Pre-treatment knowledge and depression

The results revealed a significant negative correlation between pre-existing knowledge (weighted scoring) and the change score of the BDI-II. The correlation was of medium strength (r = −0.38; p = .048). This suggest that clients who scored higher in knowledge and were confident in their ratings at pre-treatment improved less on BDI-II than those who had a lower weighted knowledge score before treatment. We controlled for severity of depression at pre-treatment, experiences of earlier treatment and the number of completed modules, which did not change the correlation. The correlation between raw knowledge scores and change on the BDI-II did not reach statistical significance (r = −0.29; p = .136). Raw knowledge scores did however predict raw BDI-II scores post-treatment, (r = 0.45, p = .016). Further, a significant, negative association was found between pre-treatment knowledge and change on the PHQ-9, in this case both for raw knowledge scores (r = −0.39; p = .039) and weighted knowledge scores (r = −0.43; p = .022). Taken together, the results suggest that being more knowledgeable prior to treatment might be predictive of poorer treatment outcome in depressive symptoms.

4. Discussion

4.1. General discussion

This is the first controlled study investigating the role of explicit knowledge in ICBT for adolescent depression. The aim of the present study was to highlight knowledge gain as an outcome in ICBT. We constructed a knowledge test dealing with depression, comorbid anxiety, and its treatment and found that clients gained knowledge including ratings of certainty of that knowledge. The treatment resulted in superior improvements in confidence-weighted knowledge, with a large between-group effect size (d = 1.25). We also tested if knowledge acquisition was related to changes in depressive symptoms, which was not the case. Finally, we tested.

if pre-existing knowledge would influence treatment outcome and found a negative association, which is discussed further below.

Improvements in knowledge and becoming more certain about one's knowledge during treatment are in line with previous studies (Andersson et al., 2012; Friedberg et al., 1998; Scogin et al., 1998; Strandskov et al., 2018). These are promising results as educational components are a central component of ICBT. The findings in the current study should however be interpreted with some caution. Participants in the treatment condition answered, on average, only about 1.4 more questions correctly compared to the control group. Unweighted knowledge scores at baseline were also relatively high (approximately 75% of questions answered correctly), suggesting that the test may have been too easy. Hence, we do not know if this relatively small increase in knowledge can be viewed as important, or if it is rather certainty of knowledge that is the key factor, as the effect size for the weighted scores was much higher.

The fairly high accuracy rate in unweighted knowledge prior treatment could also highlight the potential common-sense aspect of CBT, where the right alternative could be more or less obvious to clients even prior to treatment. This might justify the incorporation of certainty ratings. As mentioned earlier, it can be of importance to separate knowing from guessing, assuring that clients are certain about the right answers rather than guessing or being certain about the wrong answers. As pinpointed by researchers evaluating scaled-based measured on mental health literacy, there is a need of valid distinctions that separates what individuals actually know from their beliefs (O'Connor et al., 2014).

Also, when confidence is incorporated in the knowledge scores, the test might capture a sense of cognitive style, affecting the participants' level of motivation, attitudes or avoidance-tendencies during treatment. In other words: knowing less or being less confident about what one knows might constitute a tendency to be more open to change during the course of therapy. At baseline, lower scores could also reflect depression and negative attitudes with regards to capacity to be certain.

Results showed that improvements in knowledge did not correlate with improvements in depressive symptoms. These results are in line with what has been suggested in previous studies; that knowledge and confidence in knowledge alone is insufficient to explain improvement in symptoms (Andersson et al., 2012; Strandskov et al., 2018). One explanation for the lack of correlation between change in knowledge and change on the BDI-II could be the type of knowledge that the knowledge test used in this trial measure. The test used here and similar tests in other studies most likely tap explicit, declarative knowledge and relies on recognition when reading the test item. Just because clients learn and become more confident in their explicit knowledge it does not imply that they fully understand and apply this knowledge. The importance of measuring a more practical form of knowledge is supported by Bendelin et al. (2011) and Terides et al. (2016). Using qualitative methods Bendelin et al. (2011) found that clients who only took part (or rather just read texts) of psychoeducation improved less than those who applied the skills in real life. By measuring the use of practical skills, Terides et al. (2016) found significant associations between self-rated skill usage and therapeutic change for clients with anxiety and depression.

The role and evaluation of explicit, declarative knowledge is crucial to understand and evaluate further, since ICBT and CBT value psychoeducational interventions and aim to make clients well-informed. Other researchers highlight the importance of finding alternative ways to measure knowledge that captures the capacity of generalization and application of what you learn (Gumport et al., 2015; Friedberg et al., 1998). For example, using open-ended questions measuring free recall performance might better predict the ability to recall and use treatment information in one's life, or the specific content recalled might reflect more personalized treatment information than that assessed by general knowledge measures.

Interestingly, the results indicated that pre-existing knowledge predicted negative change scores in depressive symptoms immediately after treatment. These results highlight the potential importance of knowledge level prior treatment. One hypothesis is that pre-existing knowledge could be an important predictor of treatment outcome overall. Clients who lack knowledge about their condition and its treatment might be more open for change, i.e. be more curious and experience more insights than clients who already know and are certain at the outset of treatment. Patients with high pre-treatment knowledge might explicitly know and be confident about the psychoeducational points and principles of ICBT, but somehow not engage in successful application or behavioural change during treatment. For some clients, learning explicit knowledge might even prevent them from performing valued behaviours, as they might get stuck in intellectualisation instead of therapeutic change processes. It is of importance to investigate why and when ICBT does not work, since some patients might benefit less from ICBT in its current form and even deteriorate (Rozental et al., 2014). Confounding variables could potentially explain this correlation, such as severity of depression pre-treatment, number of completed modules or earlier treatment. However, controlling for these variables did not alter the correlation between pre-treatment knowledge and change in depressive symptoms. This indicates that prior explicit knowledge might be a different construct. Given the small sample size of the present study, the results should be interpreted with caution and further research is needed. In light of the risk of treatment failure it is important to identify if pre-existing knowledge is a consistent predictor of outcome in ICBT for depression in adolescents.

4.2. Strengths and limitations

One strength of the study was the high adherence with few drop-outs during treatment. Adherence was high compared with previous ICBT studies (Hilvert-Bruce et al., 2012). This study has, however, several limitations. The knowledge test had a fairly low internal consistency and was heterogeneous, assessing several idiosyncratic items rather than an integrated construct. A more elaborated test construction procedure would most likely be needed to increase reliability. Moreover, a more reliable measure would probably result in stronger findings as low alpha introduces noise. There is also room for improvement in how to measure knowledge about basic concepts, central points and ideas of ICBT. For example, by mapping the items more directly to the treatment content. The test might also need items with less obvious response options. Further, the results should be interpreted with caution since the study is underpowered and should be replicated with a larger sample. Another limitation is that we used the measure of knowledge both as a predictor and as an treatment outcome. Finally, a majority of the clients were females and went to school (94%), which could question the extern validity of the study and the possibility to generalize the results to male adolescent clients.

5. Future directions

Our study suggests that knowledge and learning in ICBT is an important topic to explore further. It would be interesting to assess procedural knowledge and application of knowledge through case scenarios or vignettes, since those measures of knowledge might better capture clients' understanding of CBT-principles and reflect personalized treatment information that they use in their everyday life. Furthermore, ICBT could be used for experimental manipulations of learning mechanisms (Dong et al., 2017), for example by randomly assigning participants to psychoeducational texts with or without emphasis on application of the information. Explicit, declarative knowledge could also be important in the long run, as indicated by Kronmüller et al. (2007). Clients might need time to use and apply the knowledge they have acquired during treatment. Clients who remember important treatment points later on might benefit by applying them when problems reoccur (for example refrain from negative thinking). Therefore, follow-up of the results are needed to investigate this possibility. Finally, knowledge could be a potential mediator of treatment outcome, as clients probably first learn something in ICBT and then apply the knowledge. One way to investigate this would be to study knowledge as a process with repeated measures during the treatment period.

In conclusion, this is the first study to evaluate the role of knowledge in a treatment for adolescent depression, indicating that explicit knowledge about ones' condition and its treatment increases.

The following is the supplementary data related to this article.

Appendix 1

Knowledge test.

mmc1.docx (31.6KB, docx)

Acknowledgments

Acknowledgements

The authors would like to thank George Vlaescu for his excellent work with the website. Kristin Silfvernagel provided valuable comments on an early version of the manuscript. Finally, we would like to thank the clinicians, counselors, and participants involved in the project.

Funding

This study was supported in part by a grant from the Swedish Foundation for Humanities and Social Sciences, grant no. P16-0883:1 (from the Swedish Central Bank) and in part by a Professors grant from Linköping University (Prof. Andersson, PI). The founding source had no involvement in design, execution or report of the study.

The authors have declared that no conflict of interest exists.

References

  1. American Psychiatric Association . American Psychiatric Press; Washington, DC: 2000. Diagnostic and Statistical Manual of Mental Disorders (4th Ed., Text Revision Ed.) [Google Scholar]
  2. Andersson G. Internet-delivered psychological treatments. Annu. Rev. Clin. Psychol. 2016;12:157–179. doi: 10.1146/annurev-clinpsy-021815-093006. [DOI] [PubMed] [Google Scholar]
  3. Andersson G., Carlbring P., Furmark T. Therapist experience and knowledge acquisition in internet-delivered CBT for social anxiety disorder: a randomized controlled trial. PLoS One. 2012;7:1–11. doi: 10.1371/journal.pone.0037411. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Andersson G., Carlbring P., Ljótsson B., Hedman E. Guided internet-based CBT for common mental disorders. J. Contemp. Psychother. 2013;43:223–233. [Google Scholar]
  5. Bendelin N., Hesser H., Dahl J., Carlbring P., Zetterqvist Nelson K., Andersson G. Experiences of guided internet-based cognitive-behavioural treatment for depression: a qualitative study. BMC Psychiatry. 2011;107:1–10. doi: 10.1186/1471-244X-11-107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brewin C.R. Theoretical foundations of cognitive-behavioral therapy for anxiety and depression. Annu. Rev. Psychol. 1996;47:33–57. doi: 10.1146/annurev.psych.47.1.33. [DOI] [PubMed] [Google Scholar]
  7. Christensen H., Griffiths K.M., Jorm A.F. Delivering interventions for depression by using the internet: randomised controlled trial. BMJ. 2004;328:265. doi: 10.1136/bmj.37945.566632.EE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Dong L., Lee J.Y., Harvey A.G. Do improved patient recall and the provision of memory support enhance treatment adherence? J. Behav. Ther. Exp. Psychiatry. 2017;54:219–228. doi: 10.1016/j.jbtep.2016.08.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Ebert D.D., Zarski A.C., Christensen H., Stikkelbroek Y., Cuijpers P., Berking M., Riper H. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One. 2015;10 doi: 10.1371/journal.pone.0119895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Fox M.P. A systematic review of the literature reporting on studies that examined the impact of interactive, computer-based patient education programs. Patient Educ. Couns. 2009;77:6–13. doi: 10.1016/j.pec.2009.02.011. [DOI] [PubMed] [Google Scholar]
  11. Friedberg R.D., Viglione D.J., Fidaleo R.A., Celeste B.L., Lovette J., Street G., Yerka E., Bieraugel M., Dumas M., Beal K.G. Measuring how we preach what we practice: psychoeducational change in depressed inpatients. J. Ration. Emot. Cogn. Behav. Ther. 1998;16:45–59. [Google Scholar]
  12. Gonzalez-Pinto A., Gonzalez C., Enjuto S., Fernandez de Corres B., Lopez P., Palomo J., Gutierrez M., Mosquera F., Perez de Heredia J.L. Psychoeducation and cognitive-behavioral therapy in bipolar disorder: an update. Acta Psychiatr. Scand. 2004;109:83–90. doi: 10.1046/j.0001-690x.2003.00240.x. [DOI] [PubMed] [Google Scholar]
  13. Griffiths K.M., Christensen H., Jorm A.F., Evans K., Groves C. Effect of web-based depression literacy and cognitive–behavioural therapy interventions on stigmatising attitudes to depression. Br. J. Psychiatry. 2004;185:342–349. doi: 10.1192/bjp.185.4.342. [DOI] [PubMed] [Google Scholar]
  14. Gumport N.B., Williams J.J., Harvey A.G. Learning cognitive behavior therapy. J. Behav. Ther. Exp. Psychiatry. 2015;48:164–169. doi: 10.1016/j.jbtep.2015.03.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Haladyna T.M. Lawrence Erlbaum Associates, Inc; Hillsdale, New Jersey: 1994. Developing and validating multiple-choice test items. [Google Scholar]
  16. Harvey A., Lee J., Williams, Hollon S., Walker M., Thompson M., Smith R. Improving outcome of psychosocial treatments by memory and learning. Perspect. Psychol. Sci. 2014;6:161–179. doi: 10.1177/1745691614521781. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hilvert-Bruce Z., Rossouw P.J., Wong N., Sunderland M., Andrews G. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav. Res. Ther. 2012;50:463–468. doi: 10.1016/j.brat.2012.04.001. [DOI] [PubMed] [Google Scholar]
  18. Johansson R., Sjöberg E., Sjögren M., Johnsson E., Carlbring P., Andersson T.…Andersson G. Tailored vs. standardized internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial. PLoS One. 2012;7 doi: 10.1371/journal.pone.0036905. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kroenke K., Spitzer R.L., Williams J.B. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kronmüller K.-T., Victor D., Schenkenbach C., Postelnicu I., Backenstrass M., Schröder J., Mundt C. Knowledge about affective disorders and outcome of depression. J. Affect. Disord. 2007;104:155–160. doi: 10.1016/j.jad.2007.02.010. [DOI] [PubMed] [Google Scholar]
  21. Lukens E.P., McFarlande W.R. Psychoeducation as evidence-based practice: considerations for practice, research and policy. Brief Treat. Crisis Interv. 2004;4:205–225. [Google Scholar]
  22. Moore D.A., Healy P.J. The trouble with overconfidence. Psychol. Rev. 2008;115:502–578. doi: 10.1037/0033-295X.115.2.502. [DOI] [PubMed] [Google Scholar]
  23. O'Connor M., Casey L., Clough B. Measuring mental health literacy–a review of scale-based measures. J. Ment. Health. 2014;23(4):197–204. doi: 10.3109/09638237.2014.910646. [DOI] [PubMed] [Google Scholar]
  24. Rozental A., Andersson G., Boettcher J., Ebert D., Cuijpers P., Knaevelsrud C., Ljótsson B., Kaldo V., Titov N., Carlbring P. Consensus statement on defining and measuring negative effects of internet interventions. Internet Interv. 2014;1:12–19. [Google Scholar]
  25. Ryhänen A.M., Siekkinen M., Rankinen S., Korvenranta H., Leino-Kilpi H. The effects of internet or interactive computer-based patient education in the field of breast cancer: a systematic literature review. Patient Educ. Couns. 2010;79:5–13. doi: 10.1016/j.pec.2009.08.005. [DOI] [PubMed] [Google Scholar]
  26. Sajatovic M., Chen P., Dines P., Shirley E.R. Psychoeducational approaches to medication adherence in patients with bipolar disorder. Dis. Manag. Health Out. 2007;15(3):181–192. [Google Scholar]
  27. Salim A., Mackinnon A., Christensen H., Griffiths K. Comparison of data analysis strategies for intent-to-treat analysis in pre-test–post-test designs with substantial dropout rates. Psychiatry Res. 2008;160:335–345. doi: 10.1016/j.psychres.2007.08.005. [DOI] [PubMed] [Google Scholar]
  28. Saunders J.B., Aasland O.G., Babor T.F., De la Fuente J.R., Grant M. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption-II. Addiction. 1993;88(6):791–804. doi: 10.1111/j.1360-0443.1993.tb02093.x. [DOI] [PubMed] [Google Scholar]
  29. Scogin F., Jamison C., Floyd M., Chaplin W. Measuring learning in depression treatment: a cognitive bibliotherapy test. Cogn. Ther. Res. 1998;22:475–482. [Google Scholar]
  30. Shapiro A.M. How including prior knowledge as a subject variable may change outcomes of learning research. Am. Educ. Res. J. 2004;41(1):159–189. [Google Scholar]
  31. Sheehan D.V., Lecrubier Y., Sheehan K.H., Amorim P., Janavs J., Weiller E., Hergueta T., Baker R., Dunbar G.C. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J. Clin. Psychol. 1998;59:22–57. [PubMed] [Google Scholar]
  32. Silfvernagel K., Gren-Landell M., Emanuelsson M., Carlbring P., Andersson G. Individually tailored internet-based cognitive behavior therapy for adolescents with anxiety disorders: a pilot effectiveness study. Internet Interv. 2015;2:297–302. doi: 10.1016/j.invent.2017.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Steer R.A., Kumar G., Ranieri W.F., Beck A.T. Use of the Beck Depression Inventory-II with adolescent psychiatric outpatients. J. Psychopathol. Behav. Assess. 1998;20:127–137. [Google Scholar]
  34. Strandskov S.W., Ghaderi A., Andersson H., Parmskog N., Hjort E., Svanberg Wärn A., Jannert M., Andersson G. Effects of tailored and ACT-influenced internet-based CBT for eating disorders and the relation between knowledge acquisition and outcome: a randomized controlled trial. Behav. Ther. 2018;48:624–637. doi: 10.1016/j.beth.2017.02.002. (in press) [DOI] [PubMed] [Google Scholar]
  35. Terides M.D., Dear B.F., Karin E., Jones M.P., Gandy M., Fogliati V.J.…Titov N. The frequency of actions and thoughts scale: development and psychometric validation of a measure of adaptive behaviours and cognitions. Cogn. Behav. Ther. 2016;45:196–216. doi: 10.1080/16506073.2016.1149876. [DOI] [PubMed] [Google Scholar]
  36. Thapar A., Collishaw S., Pine D.S., Thapar A.K. Depression in adolescence. Lancet. 2012;379:1056–1067. doi: 10.1016/S0140-6736(11)60871-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Tiedens L.Z., Linton S. Judgment under emotional certainty and uncertainty: the effects of specific emotions on information processing. J. Pers. Soc. Psychol. 2001;81(6):973. doi: 10.1037//0022-3514.81.6.973. [DOI] [PubMed] [Google Scholar]
  38. Topooco N., Berg M., Johansson S., Liljethörn L., Radvogin R., Nordgren L., Zetterqvist M., Andersson G. The efficacy of chat- and internet-based cognitive behaviour therapy for adolescent depression: a randomised controlled trial. BJPsych Open. 2018;4:199–207. doi: 10.1192/bjo.2018.18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Tursi M.F., von Werne Baes C., de Barros Camacho F.R., de Carvalho Tofoli S.M., Juruena M.F. Effectiveness of psychoeducation for depression: a systematic review. Aust. N. Z. J. Psychiatry. 2013;47:1019–1031. doi: 10.1177/0004867413491154. [DOI] [PubMed] [Google Scholar]
  40. Vickers A.J., Altman D.G. Analysing controlled trials with baseline and follow up measurements. BMJ. 2001;323(7321):1123–1124. doi: 10.1136/bmj.323.7321.1123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Vlaescu G., Alasjö A., Miloff A., Carlbring P., Andersson G. Features and functionality of the Iterapi platform for internet-based psychological treatment. Internet Interv. 2016;6:107–114. doi: 10.1016/j.invent.2016.09.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Zuckerbrot R.A., Cheung A.H., Jensen P.S., Stein R.E., Laraque D. Guidelines for Adolescent Depression in Primary Care (GLAD-PC): I. identification, assessment, and initial management. Pediatrics. 2007;120:1299–1312. doi: 10.1542/peds.2007-1144. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix 1

Knowledge test.

mmc1.docx (31.6KB, docx)

Articles from Internet Interventions are provided here courtesy of Elsevier

RESOURCES