Facilitating and hindering factors in Internet-delivered treatment for insomnia and depression

Abstract

Insomnia and depression is a common and debilitating comorbidity, and treatment is usually given mainly for depression. Guided Internet-based cognitive behavioral therapy for insomnia (ICBT-i) was, in a recent study on which this report is based, found superior to a treatment for depression (ICBT-d) for this patient group, but many patients did not reach remission.

Aims

To identify facilitating and hindering factors for patients in ICBT-i and ICBT-d and formulate hypotheses for future research.

Method

Qualitative telephone interviews at the time of the 6-month follow-up. Thirty-five interviews were done and analyzed with a grounded theory approach. Based on the qualitative results, an iterative method-triangulation including quantitative and semi-qualitative was performed.

Results

The interviews were coded into 738 sentences, condensed into 47 categories and finally 11 themes. Four areas were investigated further with method triangulation: Opinions about treatment, adherence, hindering symptoms and acceptance. Patients in ICBT-i were more positive regarding the treatment than patients in ICBT-d. Using treatment components was positively associated with outcome in both groups. Symptoms of insomnia, depression and other comorbidities were perceived as more hindering for ICBT-d than for ICBT-i. Acceptance of diagnose-related problems as well as negative emotions and cognitions was positively associated with outcome for ICBT-i.

Proposed future research hypotheses

1) A combination of CBT for insomnia and CBT for depression is more effective than only one of the treatments. 2) Additional therapist support increases outcomes for patients with more comorbidities. 3) Acceptance is a mechanism of change in CBT-i.

Highlights

• •We performed a qualitative study, interviewing 35 participants in an RCT of Internet-delivered CBT for insomnia or depression
• •Participants had comorbid insomnia and depression
• •Qualitative analyses were deepened with quantitative and semi-qualitative data
• •Participants in the insomnia treatment were more positive and less bothered by comorbid symptoms
• •The study generated hypotheses for future research: combination treatment, therapist support and acceptance as a mechanism

1. Introduction

Insomnia and major depression are probably the two most common psychiatric diagnoses, with a prevalence of around 10–20 and 5–8% respectively (Ohayon and Roth, 2003, Kim et al., 2000, Ford and Kamerow, 1989, Young et al., 2008, Wittchen and Jacobi, 2005). They both cause a lot of suffering for the individual and a great cost to society (Daley et al., 2009, Bijl and Ravelli, 2000). Insomnia is often more or less chronic if left untreated, and depression is likely to recur after the first episode (Burcusa and Iacono, 2007, Ford and Kamerow, 1989). Comorbidity between insomnia and depression is very common, with approximately two thirds of depressed patients also suffering from insomnia (Buysse et al., 2008, Soldatos, 1994).

The most common treatment for both conditions is medication. There are effective psychological treatments, though generally more difficult to come by. The psychotherapy form with the most convincing scientific evidence for mild to moderate depression is cognitive behavioral therapy (CBT) (Butler et al., 2006). For insomnia, CBT is considered the treatment of choice, and has shown long-term effects superior to that of sleep medication (Morin et al., 2006, Riemann and Perlis, 2009).

The most common approach to treating comorbid insomnia and depression is to treat the depression. Depression has historically often been seen as the cause of the sleeping problems, and treating depression is expected to improve sleep. Previous studies show, however, that insomnia often precedes depression (Walsh, 2004), and that untreated insomnia increases the risk of relapse into depression (Perlis et al., 1997). In a previous randomized controlled trial (RCT) (Blom et al., 2015a) of guided Internet-based cognitive behavioral therapy (ICBT), we found that ICBT for insomnia (ICBT-i) was more effective than CBT for depression (ICBT-d) for this patient group, but also that patients with comorbid insomnia and depression did not improve as much as expected, when compared to patients with one of the diagnoses (Hedman et al., 2013, Jernelöv et al., 2012, Blom et al., 2015b). This sample seemed to be more burdened than patients with one of the diagnoses.

In order to understand more about how to improve the treatment, we wanted to formulate testable hypotheses regarding facilitating and hindering factors for these patients in their work with ICBT, and we decided to do this in a qualitative study, based on the previous RCT (Blom et al., 2015a) with the possibility to use available quantitative data to shed further light on qualitative findings.

Previous qualitative studies examining randomized clinical trials of psychotherapy are few, and those that exist take different views and perspectives. A recently published review examined qualitative studies of insomnia and the current state of knowledge (Araújo et al., 2016). This review looks into the experience of insomnia and insomniacs' views on treatment. Among other things, they conclude that insomniacs are a frustrated group with insomnia affecting life 24/7. Insomniacs do not find that they are being fully understood by health care providers, who seem largely unaware of non-pharmacological treatment options.

There is an increasing amount of qualitative studies looking at ICBT. The Internet is a fairly new way of disseminating psychotherapy, and there is an urge to learn more about how and why it works. Some of the previous qualitative studies on ICBT look into motivators and motivations (Donkin and Glozier, 2012), expectations and experiences in primary care (Beattie et al., 2009) and therapist behaviors (Paxling et al., 2013, Holländare et al., 2016). A study on ICBT for depression (Bendelin et al., 2011) found that the process of change corresponded to theories of change in face-to-face therapy, and that patients who attribute success to themselves and take responsibility for their treatment benefit more. The report that is perhaps most relevant to our study is about patients' experience of helpfulness in ICBT for depression (Lillevoll et al., 2013), even though the therapist support in this study was face-to-face, making it more of a blended therapy. That study emphasizes active engagement of the patient, guidance from the therapist and the content of the treatment as the most helpful dimensions.

The aim of the present study was to investigate factors that hinder and facilitate the work with ICBT for insomnia or depression, for patients with both diagnoses, by means of qualitative telephone interviews and method triangulation using interview data, quantitative data and semi-qualitative clinical data from the Internet treatment platform. The findings were used to formulate research hypotheses that, if later evidenced, would help improve treatment for this patient group.

2. Method

This study was conducted in parallel with the 6-month follow-up of an RCT comparing guided Internet-delivered CBT for insomnia (ICBT-i) to guided Internet-delivered CBT for depression (ICBT-d) for participants diagnosed with both insomnia and major depression (Blom et al., 2015a). The RCT was registered at Clinicaltrials.gov, registration ID: NCT01256099 and was set at the Internet Psychiatry Clinic, Stockholm County public health, Sweden.

2.1. Description of the original RCT

2.1.1. Design

The design of the original study was a nine week randomized controlled trial with six-, twelve- and 36-month follow-up (the 36-month follow-up results have not yet been reported). Participants were 43 adults diagnosed with comorbid insomnia and depression, recruited via media and assessed by psychiatrists. The study was advertised as being directed at individuals with both insomnia and depression, and prior to consenting, participants were informed that they would be randomized to either treatment for insomnia or treatment for depression. Randomization was carried out by an independent person using www.random.org.

2.1.2. Outcome measures

Primary outcome measures were the symptom self-rating scales Insomnia Severity Index, ISI (Morin et al., 2011) and Montgomery Åsberg Depression Rating Scale MADRS-S (Svanborg and Åsberg, 1994), assessed before and after treatment with follow-up after 6, 12 and 36 months. The participants' use of sleep medication and need for further treatment after completion of ICBT were also investigated. Mean (SD) ISI-score pre-treatment was 19 (4) and mean (SD) MADRS-S score was 26 (6).

2.1.3. Interventions and support

Interventions were ICBT for either insomnia (ICBT-i) or depression (ICBT-d). The treatments were delivered on the same technical platform and accessed on a secure web site which only the participant and their therapist could access. The modules consisted of text to read, questions to answer on theory, behavioral change exercises, work sheets, and for the ICBT-i group a sleep diary. Participants were expected to complete on average one module per week. Each module ended with the participant sending in a home-work report via a secure messaging system. The therapist received the report, reviewed answers to homework questions, work sheets that were filled out and sleep diary (ICBT-i only), gave written feedback within 24 h on week days and finally gave the participant access to the next module. The participants also had the possibility to send messages with questions or comments to their therapist. Therapists were instructed not to give advice that was outside the scope of the manual. If the participants were inactive for one week, a mobile text message was sent by the therapist. If there still was no activity, a phone call was made, and if these attempts at contact failed for around three weeks, a letter was sent, encouraging the participant to make contact. The six therapists were final (fifth) year students of clinical psychology at master level, with at least 18 months of theoretical and practical supervised training in CBT, who were supervised by a licensed clinical psychologist with CBT and insomnia/depression treatment expertise.

The insomnia treatment consisted of standard CBT-i components in a manual previously tried in several trials (Blom et al., 2015b, Jernelöv et al., 2012, Kaldo et al., 2015a): psychoeducation about sleep and CBT-i, sleep hygiene, education on sleep medication and how to quit, sleep restriction and stimulus control, stress management and reappraisal of negative thoughts about sleep. The main focus during treatment was on sleep restriction and stimulus control.

The depression treatment was previously tried both in an RCT and regular care, where it is currently in use (Andersson et al., 2005, Hedman et al., 2013) and consisted of psychoeducation on depression and CBT, behavioral activation, cognitive reappraisal and coping strategies for handling anxiety and worry. The treatment and therapist support focused on behavioral activation and reappraisal of negative thoughts.

More information about the RCT can be found in the original article (Blom et al., 2015a).

2.2. Procedure

At the time of the 6-month assessment (FU6) of the RCT, all participants were contacted by phone and asked if they wanted to participate in an extended interview about the treatment. Thirty-seven of the 43 participants were reached and all agreed to participate. All 37 were interviewed but the recordings were lost due to technical problems for two of them, leaving data from 35 interviewed participants, 18 from ICBT-i and 17 from ICBT-d. Mean (SD) age at baseline was 48 (13) years and mean number of years with insomnia was 16 (SD 14). There were 51% females, 54% of patients used sleep medication and 37% used antidepressants in the two weeks prior to assessment.

The interviews were done via telephone and recorded with a tape recorder, then transcribed verbatim by a medical secretary at the clinic. The first author (KB) did 28 interviews and the second author (SJ) seven.

The interviews were based on two questions, starting with the question: “How did you think the treatment went?”, asking the participants to choose between the alternatives very well, rather well, rather badly or very badly. The second question was: “Why do you think it went [well/badly]?”. After this the interview was free to follow whatever the participant brought up, trying to stay with open questions to allow for variation in the participants view of the treatment. The interviewer had some additional themes they could bring into the interview, if the participants ran out of things to say or needed help to remember:

• a)problems or obstacles during treatment?
• b)the treatment texts: extent and comprehensibility
• c)technology
• d)issues with getting started
• e)routines for working with treatment
• f)making use of treatment methods
• g)effects of using methods
• h)the treatment's relevance to the participants problems
• i)therapist support
• j)expectations before treatment, and after some time in treatment

2.3. Interviewers and analysts

The interviews and analyses were performed by the first and second authors. The first author (KB) is a licensed psychologist with a master of science in psychology. She has been studying and working at a public health-run psychiatric clinic with insomnia and depression treatment (CBT) for approximately eight years and is in the final stages of her PhD-studies within this area at Karolinska Institutet, Stockholm, Sweden. KB was actively involved in the original study on which this report is based, and her experiences of supervising the treatments and coordinating the project as a whole have influenced the design of this study, e.g. the interview questions, as well as the analyses. The second author (SJ) is a licensed psychologist and PhD at Karolinska Institutet, working there as a researcher and lecturer for the psychologist education program. She has been studying and working with insomnia and stress both in clinical practice (with CBT) and in the context of studies for approximately 16 years.

2.4. Analyses

The analysis of the interviews was based on the methodology from grounded theory as described by Guvå and Hylander (2003). To enrich the results, we used method-triangulation (Flick, 2004) of some of the interview material, by doing quantitative and semi-qualitative analyses described further below. First, the transcribed material was read carefully and coded into sentences representing everything in the interviews deemed meaningful by the coder (first author, KB). Second, these sentences (n = 738) were in turn condensed into 47 categories, encompassing sentences with similar content. Third, the full interviews were re-read and assigned the categories that matched the content, by the first author. The second author (SJ) also performed the third step, for 6 participants, in order to test the inter-rater reliability of the coding and categories. The inter-rater-reliability testing should be considered as a test of the soundness, or face-validity, of the generation of categories. Finally, as a fourth step, the categories were condensed further into broader themes.

While doing the coding and re-coding in the four steps described above, as well as during the post-hoc analyses, memos were taken on the authors' reflections regarding the material. The memos were used both as notes on which findings should be used and presented in the report, and as input to which quantitative and semi-qualitative data that should be considered and analyzed in order to complement the understanding of the material, thus providing data for method-triangulation. Each step in the analyses was used as input to new analyses, both qualitative, quantitative and semi-qualitative, in an iterative manner until the analyses reached saturation, i.e. when all the available data of relevance, both qualitative and quantitative, had been analyzed. As such, all these extra analyses are by nature post-hoc. Note that the saturation principle was not used when deciding on how many subjects to interview, only to decide which additional semi-qualitative and quantitative analyses to do. Themes from the interviews that were not chosen for further analysis, are only presented in Table 3, where the reader can see which interview categories constitute a theme, and by how many participants they have been expressed. The details of the quantitative and semi-qualitative measures are found in the Results section, since choosing measures and data sources was part of the analytical process.

Table 3.

Number of participants making statements in each category, divided by group and effect. Themes and which categories make up each theme.

Theme	Total n = 35		Above average treatment gain		Below average treatment gain
Category	ICBT-i n = 18	ICBT-d n = 17	ICBT-i upper n = 9	ICBT-d upper n = 9	ICBT-i lower n = 9	ICBT-d lower n = 8
1 Problems with ICBT content	5	11	1	3	4	8
1.1 Too much material	2	5	1	2	1	3
1.2 Material too difficult	2	3	0	1	2	2
1.3 Difficult with text based material	0	2	0	0	0	2
1.4 Difficult to work via Internet	1	1	0	0	1	1
2 Stressful demanding treatment	14	14	5	7	9	7
2.1 Difficult and demanding treatment	6	3	2	2	4	1
2.2 Lack of time	2	5	1	2	1	3
2.3 Stressed by performance expectations/lagging behind	3	4	1	2	2	2
2.4 Feelings of failure/guilt from poor performance	3	2	1	1	2	1
3 Hindered by insomnia/depression symptoms	6	13	0	3	6	10
3.1 Difficulties getting started	1	3	0	1	1	2
3.2 Diagnosis-related problems when working with treatment	2	5	0	1	2	4
3.3 Feelings of being tired	2	5	0	1	2	4
3.4 Worry or stress due to sleep deprivation	1	0	0	0	1	0
4 Hindered by illness in general	1	9	0	4	1	5
4.1 Somatic illnesses	1	4	0	2	1	2
4.2 Somatic pain	0	3	0	1	0	2
4.3 Primary diagnosis unclear to patient	0	1	0	1	0	0
4.4 On sick leave — hindering	0	1	0	0	0	1
5 Negative towards treatment	17	18	4	9	13	9
5.1 Initially positive to treatment but changed to negative	1	1	0	0	1	1
5.2 Negative/skeptical towards treatment from start	3	2	1	1	2	1
5.3 Nothing new in treatment	2	1	0	1	2	0
5.4 Wanted the other treatment — depression	5	0	1	0	4	0
5.5 Wanted the other treatment — insomnia	0	6	0	2	0	4
5.6 Negative attitude due to misunderstanding about randomization	1	2	0	1	1	1
5.7 Insufficient support	5	6	2	4	3	2
6 Positive towards treatment	23	19	16	12	7	7
6.1 Likes Internet format	9	7	5	4	4	3
6.2 Positive towards treatment from start	4	1	4	1	0	0
6.3 Initially negative to treatment but changed to positive	3	2	3	2	0	0
6.4 Psychoeducation seen as important	0	2	0	0	0	2
6.5 Flexibility as to where and when (treatment)	2	0	1	0	1	0
6.6 Good support from therapist	5	5	3	3	2	2
6.7 Material difficult — positive	0	2	0	2	0	0
7 Active with treatment components	21	20	13	10	8	10
7.1 Starts to do physical exercise	0	2	0	0	0	2
7.2 Behavioral activation other than physical exercise	0	5	0	1	0	4
7.3 Appreciates working with negative thoughts	0	3	0	2	0	1
7.4 Motivated/engaged in treatment	8	6	5	4	3	2
7.5 Adherent to treatment in general	7	4	5	3	2	1
7.6 Did sleep restriction	6	0	3	0	3	0
8 Inactive with treatment components	2	3	0	1	2	2
8.1 Active but in unhelpful way or with unhelpful things	1	0	0	0	1	0
8.2 Not motivated	0	2	0	1	0	1
8.3 Reflects but does not act	1	1	0	0	1	1
9 Acceptance of diagnosis-related problems	7	0	5	0	2	0
9.1 Acceptance of diagnosis-related problems	7	0	5	0	2	0
10 Facilitating other	9	3	7	2	2	1
10.1 On sick leave — facilitating	1	0	0	0	1	0
10.2 Handling of medication — facilitating	1	1	1	1	0	0
10.3 Positive thanks to success	7	2	6	1	1	1
11 Hindering other	4	3	1	1	3	2
11.1 Other problems in life	2	1	1	0	1	1
11.2 Bothered by questionnaires and measurements	0	1	0	0	0	1
11.3 Handling of medication — negative	1	0	0	0	1	0
11.4 Expresses non-acceptance of sleeping problems	1	1	0	1	1	0
Initial question:
Went very or rather well	13a	7	8 a	4	5	3
Went very or rather badly	4a	10	0 a	4	4	6

^aOne participant in ICBT-i upper refused to state an opinion.

In order to facilitate comparison of the treatment outcomes over both the main outcome measures (ISI and MADRS-S) and the post- and FU6-assessments, we calculated a combined measure: Z-change. Since the data used in our analyses originated from both the post-assessments and the six month follow-up, we decided to include both assessment points when calculating Z-change, even though the interpretation of the value is made less transparent due to the participant's variations between the assessment points. First, we calculated a mean follow-up score (FU) for the post and 6-month follow-up (FU6) scores for each participant and measurement (i.e. mean FU ISI-score = (ISI (post) + ISI (FU6)) / 2 and mean FU MADRS-S-score = (MADRS-S (post) + MADRS-S (FU6)) / 2). We then calculated a change-variable taking the pre-treatment value minus the mean FU score: ISI-change = ISI (pre) − mean FU ISI-score; MADRS-S-change = MADRS-S (pre) − mean FU MADRS-S-score. These change variables, representing the average change from before treatment, were transformed into Z-values, i.e. the change expressed as standardized deviations from the mean, enabling comparison between measures. Finally, we calculated a mean Z from the Z for ISI and Z for MADRS-S, ending up with a Z-value representing a specific patient's change from the pre- to post/FU6-assessments for both outcomes. We call this variable Z-change: a positive score indicates that the participant's average follow-up score for both ISI and MADRS-S is better than the mean for the entire sample, and a negative score indicates a follow-up score that is worse than the mean of entire sample.

When presenting quotes from the interviews, some words are added for clarity and presented [between brackets]. Each participant quoted is presented with group identity, Z-change value and an estimate of the participant's adherence, compiled of semi-qualitative data in the platform and questionnaires about the usage of main treatment components, the number of modules completed and days spent with treatment.

χ²-Tests of positive and negative opinions about the treatment were based on the sum of positive statements and the sum of negative statements (Fig. 1) from all participants in each treatment group, i.e. number of positive or negative statements coded for ICBT-i versus the number of positive or negative statements not coded for ICBT-i, compared to the same coding for ICBT-d. Statistical analyses were performed using the SPSS version 22 software package (IBM Inc., Armonk, NY, USA).

Fig. 1.

Themes and combined themes from Table 3 that were analyzed more in-depth with method triangulation. Numbers represent number of statements per subgroup. Hindered by symptoms = sum of themes 3 and 4; positive regarding treatment = sum of theme 6 and category 10.3; negative regarding treatment = sum of themes 1, 2, and 5; sum positive statements = sum of themes 6, 7, 9 and 10; sum negative statements = sum of themes 1, 2, 3, 4, 5, 8 and 11. Upper = subgroups with Z-change above average, lower = subgroups with Z-change below average.

3. Results

3.1. Definition of subgroups

The two treatment groups were divided into two halves based on Z-change, called upper and lower, based on having a Z-change above or below the mean of the group. The Z-change data for the groups can be found in Table 1. Note that a negative Z-change value is not the same as a negative treatment outcome. Detailed outcome data can be found in the original article.

Table 1.

Maximum, minimum and mean Z-change per group.

Group, n	Min Z-change/max Z-change	Mean Z-change (SD)
ICBT-i, 18	− 1.88/2.70	0.07 (1.08)
ICBT-d, 17	− 1.31/0.99	− 0.08 (0.68)
ICBT-i upper, 9	0.24/2.70	0.86 (0.80)
ICBT-d upper, 9	0.01/0.99	0.46 (0.33)
ICBT-i lower, 9	− 1.88/0.14	− 0.72 (0.66)
ICBT-d lower, 8	− 1.31/0.0	− 0.68 (0.42)

The eight participants not interviewed or properly recorded, therefore not included in the analyses, had mean Z-change values of 0.71 (SD 0.88) for ICBT-i (n = 4) and − 0.52 (SD 0.58) for ICBT-d (n = 4).

3.2. Coding, categories and themes

The procedure for coding the interviews is described in the Method section. Inter-rater reliability for associating categories to interviews was 89%. Examples of the coding of transcribed text into condensed sentences and then into categories and themes are found in Table 2.

Table 2.

Example of the coding process.

Interview text as transcribed	Condensed sentence	Category	Theme
1. I had problems getting started with it and difficulties spending the required time at the computer for this. Both reading and doing exercises. I think that because of my diagnosis I have problems getting started.	1. Difficulties reading and doing exercises	3.1 Difficulties getting started	3 Hindered by insomnia/depression symptoms
	2 Diagnosis-related difficulties getting started	3.2 Diagnosis-related problems when working with treatment
2. It felt like everything went badly, sort of… Well, it was like these anxiety symptoms and such, they kind of got in the way of concentrating completely… and sort of make it work.	3 Experiences of everything going badly
	4 Symptoms of anxiety hindered concentration

The categories derived from the interview material are presented in Table 3, showing the number of participants that said something corresponding to the category. Each participant could only be assigned a category once, even if they said the same or similar things several times during the interview.

In the following, we will present more details about the themes and categories from the interviews that were deemed most likely to satisfy the aim of the study, thus triggering more investigation. An overview of the difference between the groups regarding these themes is presented in Fig. 1.

3.3. Opinions about the treatment and therapist support

3.3.1. Qualitative analysis of opinions

Participants in both treatment groups expressed both positive and negative attitudes toward the treatments, but ICBT-i expressed more positive and less negative statements than ICBT-d (see Fig. 1). Furthermore, the ICBT-i upper participants were more positive about the treatment than the other subgroups. Both groups express some negative opinions about the treatment but as can be seen specified in Table 3, ICBT-d lower expressed more problems with the treatment content than the other groups, complaining that the material was too difficult and too much to work with, while ICBT-i lower seemed the most stressed about treatment. In ICBT-d the two subgroups seemed equally stressed from doing the treatment and ICBT-i upper is least stressed. About half of the participants expressed being positive towards the Internet format, and this was similar in both groups. Very few expressed being completely against the Internet format:

Hmm, it [the treatment] helped me and had the desired effect. [...] Yes, [I liked] that you can do it [work with treatment] at home. Do it yourself. When you have the time. (ICBT-i, Z-change: 2.7, adherence: high)

Aumm, I thought while I was inside the depression that it was very hard to adhere to all the material, it was way too heavy and too much to understand. But when I look back afterwards, when I'm out of the depression, there are some really good bits in it. So it didn't help me so much at the time, but it will help in the future. […] Well, it was these practical exercises where you should weigh the positive and the negative and think more positively, and some practical exercises in the everyday life that help me [now], I think. (ICBT-d, Z-change: 0.99, adherence: medium — high)

Yes, it was this thing when you were supposed to think through which things have positive and negative consequences and… Positive… When you should think about your inside all the time. I found that very difficult. (ICBT-d, Z-change: 0.01, adherence: high)

Several participants from both groups said in the interviews that they would have preferred the other treatment (see categories 5.5. and 5.6 in Table 3). A few participants in ICBT-d made it very clear that they viewed the depression entirely as a consequence of their insomnia (one did not agree that they had been depressed at all), and that while being interesting and educative, the depression treatment was pointless for them:

I felt I ended up in the wrong group… It is sort of… the sleep makes you depressed… when you sleep badly… sleep extremely badly and cannot handle work, it becomes… it makes you depressed. […] It was interesting to read about depression in this way, in the study, and how you can change your thinking and such, but I still don't sleep any better. (ICBT-d, Z-change: − 0.37, adherence: high)

In ICBT-i, on the other hand, the participants wanting the depression treatment complained more of a general inability to work with the treatment due to the symptoms of depression: difficulties getting started, difficulties concentrating, feelings of failure and a general low capability to work:

I think I should really have ended up in the other study [treatment], which I think maybe would have been of more use to me. If I could have been helped by that study, I might have been able to… benefit more from the study. It may sound a bit strange, but that's really how I feel, that it was mainly the depression that, sort of, hindered me from working with my sleep. (ICBT-i, Z-change: − 1.67, adherence: low)

While several participants expressed appreciation of their therapist, several also said that they would have wanted something more — that only doing treatment via the internet was not sufficient. Very few had any spontaneous ideas as to what this might be, but when asked about face-to-face sessions, group sessions or telephone support, many were partial to getting telephone support from their internet-therapist.

Hmm, well, I thought about this contact, that you were supposed to have, during treatment. […] Someone calling, like you do now, it, sort of, is much easier… if you could have that possibility, to, kind of — [imitating therapist] “I put this in you plan, how are you doing? Can you do this or are there problems?” — you know? Meet me half way, so to say. It's easier, you hesitate to get in touch, it's like: I know what to do but I can't get started. But if someone called me I and pushed me I would shape up a little. […] Once a week would be nice. Twice a week, oh shit, then you know: this is serious, ha ha! (ICBT-i. Z-change: 0.57, adherence: high)

3.3.2. Quantitative analyses of opinions

χ²-Tests confirmed that ICBT-i expressed more positive opinions than ICBT-d about the treatment (see Fig. 1). In ICBT-i, 73 out of 288 possible positive statements (18 participants times 16 categories) were made vs. 49 out of 272 in ICBT-d, χ²-value = 4.4, df = 1, p = 0.04. ICBT-d expressed more negative opinions than ICBT-i: in ICBT-d 81 out of 446 possible negative statements were made vs. 53 out of 505 in ICBT-i, χ²-value = 8.6, df = 1, p = 0.003. To investigate whether opinions about the treatment were related to outcome, we analyzed the Client Satisfaction Questionnaire (CSQ-8, ref) distributed post-treatment. The CSQ-score ranges from 0 to 32 points, with a higher score indicating more satisfaction with a treatment. Mean (standard deviation (SD), confidence interval (CI)) scores post-treatment were, for ICBT-i: 22.7 (5.3, 19.9–25.5) and for ICBT-d: 23.2 (4.4, 20.9–25.4), not a significant difference. Z-change correlated significantly with CSQ-8 for ICBT-i (r = 0.85, CI 0.64–0.95, p < 0.01) but not for ICBT-d (r = 0.48, CI − 0.002–0.78, p > 0.05).

We also wanted to look into data on the opinion expressed in the interviews, responding to the question “how did you think it went? (very well, rather well, rather badly or very badly). This opinion scale (see Table 3, where the scale has been compressed into the alternatives “well” and “badly”) correlated significantly with Z-change for ICBT-i (r = 0.78, CI 0.48–0.92, p < 0.01) but not for ICBT-d (r = 0.32, CI − 0.19–0.70, p > 0.05). We also did a Fischer exact test to check the difference between groups regarding their opinion of how the treatment went (well or badly, Table 3, initial question), and found a significant difference indicating that ICBT-i participants were more positive (Fischer exact test value = 0.047, df = 1, p < 0.05).

Participants were asked post-treatment, in a questionnaire, what type of support they would have preferred. The choices were (number of participants per group choosing this alternative within parenthesis): only support via Internet (ICBT-i: 8, ICBT-d: 5), weekly telephone support (ICBT-i: 2, ICBT-d: 6), weekly individual face-to-face session (ICBT-i: 2, ICBT-d: 6), weekly group sessions (ICBT-i: 4, ICBT-d: 0) and no contact at all (ICBT-i: 1, ICBT-d: 0).

3.4. Motivation and adherence

3.4.1. Qualitative analysis of motivation and adherence

During the interview analyses, we found that participants in both groups, from both the upper and lower subgroups, expressed that they had been motivated and engaged in the treatment, and that they felt that they had adhered to the methods in the treatment. Looking at categories 7.4 and 7.5 in Table 3, we see that slightly more participants in the above average subgroups reported being motivated and adherent to treatment, compared to the lower subgroups. All four groups expressed being active with treatment components (see Fig. 1), ICBT-i upper a little more than ICBT-i lower, no difference for ICBT-d subgroups:

(Adherence) It was important to schedule activities, I talked to my contact [therapist] about this and she said it was OK to use my calendar. I think doing what I did worked smoothly, getting it into my everyday life rather than doing a lot of extra work, so to say. […] It is important to work with my thoughts, anyway, that was really good. Yes, you got to learn a lot about yourself too I think, incredibly much. (ICBT-d, Z-change: − 1.3, adherence: very high)

(Motivation) I wanted it [to work] and I believed in it, you know. I had hoped for an improvement. (ICBT-i, Z-change: − 1.88, adherence: medium-low)

The interviews thus indicated that the level of self-reported motivation, adherence and activity, though largely similar, might have had something to do with outcome, at least in ICBT-i.

3.4.2. Quantitative analyses of motivation and adherence

We wanted to look into the treatment adherence in more detail to see if the other data sources indicated the same. This was done by analyzing a post-treatment evaluation questionnaire and the treatment platform material, for example homework reports. In the questionnaire administered post-treatment, participants were asked to score how much they had used each treatment component on a 5-point scale from “not at all” (0 points) to “very much” (4 points). Correlating total use of all treatment components to Z-change produced significant correlations with the usage of the components for both ICBT-i (r = 0.53, CI 0.07–0.81, p < 0.05) and ICBT-d (r = 0.56, CI 0.11–0.82, p < 0.05).

Complementary analyses of adherence were done by looking at the conversations between therapist and participant, as well as the completion of homework assignments, while considering the quality of this work. Using this information, we calculated number of completed modules and number of active days in treatment. When correlating Z-change to number of completed modules and number of active treatment days we found no significant correlations for any of the groups or subgroups, nor for the whole sample.

3.5. Hindering comorbidities

3.5.1. Qualitative analysis of comorbidities

ICBT-d lower stands out as the group finding themselves most hindered by their diagnoses, i.e. insomnia and/or depression, as well as by other illnesses, during treatment. Looking more closely at these themes (see Table 3), we find that somatic illness and pain have come up in the interviews with several participants in both ICBT-d subgroups, but only for one individual in ICBT-i. Also, participants in ICBT-d have expressed that they are hindered by symptoms of their insomnia and/or depression diagnoses twice as many times as participants in ICBT-i, while participants in ICBT-i upper express no such hinders in the interviews.

I think that because of my diagnosis have problems getting started with things. But I also had another complication last year which was disc herniation, and I was in a lot of pain, I could never sit still by the computer for more than five to ten minutes. It was very hard. (ICBT-d, Z-change: − 0.84, adherence: very low)

I think my theory is that there is something physiological behind [my problems] and that I have other problems too. Not only is my sleep extreme, I also have a number of other neurological problems at the same time. So I think the sleep is connected to that and also to hormonal problems. I think it's caused by something physiological and then not much else you do will work, you know. (ICBT-i, Z-change: − 0.73, adherence: high)

I never felt, and still don't feel depressed for any other reason [than sleeping problems]. I was just so, or is still, so dreadfully tired. But of course in that condition… it becomes… The whole existence becomes hard… (ICBT-d, Z-change: 0.17, adherence: medium)

It's like I said, I suffer from rheumatoid pain and I get a very low mood when I'm in a lot of pain. But that's not the same as suffering from depression, it's more connected to waking up and feeling ill, that sort of puts you in a low mood. (ICBT-d, Z-change: 0.48, adherence: low)

3.5.2. Quantitative analyses of comorbidities

We wanted to find out whether participants in ICBT-d were in fact more burdened by comorbidities than ICBT-i, or if, perhaps, the fact that they complained more about their symptoms hindering treatment could be pertaining to some characteristics of the treatment itself. We therefore analyzed the information about comorbid conditions in the screening questionnaires and assessment interviews of the original RCT. An on-line screening questionnaire asked participants about 20 specified disorders (somatic-, behavioral medicine-, psychiatric- and sleep disorders) and had room for adding other disorders. In the following screening telephone interview, questions were asked about the conditions they had reported in the questionnaire, and they were also asked about anything they might have forgotten or left out in the questionnaire. Participants were asked whether they thought they had the disorder, or whether they were diagnosed by a health care professional. The diagnoses were not verified in any other way. We compared the groups using independent t-tests on total number of diagnosed comorbid disorders. The ICBT-d group had somewhat higher number of diagnosed comorbid disorders pretreatment, but not significantly so (t = 1.2, df = 33, p = 0.2). Mean (SD, CI) number of comorbidities was for ICBT-i: 1.8 (1.9, 0.9–2.8) and for ICBT-d: 2.8 (2.6, 1.4–4.1). Analyzing the subgroups revealed that ICBT-i upper had significantly fewer comorbid disorders than ICBT-d upper (t = 2.4, df = 16, p = 0.03) and near significantly fewer comorbidities than ICBT-i lower (t = 2.0, df = 16, p = 0.06). Mean (SD, CI) number of comorbidities was for ICBT-i upper: 1.0 (0.9, 0.3–1.7); ICBT-i lower: 2.7 (2.3, 0.9–4.5); ICBT-d upper: 2.8 (2.1, 1.2–4.4); and ICBT-d lower: 2.8 (3.2, 0.08–5.4).

3.6. Acceptance

3.6.1. Qualitative analysis of acceptance

The interview material indicated that accepting the sleeping problems was an important part of what several of the participants in the ICBT-i group perceived as helpful in reaching a satisfactory result. Participants in ICBT-d did not bring up acceptance of symptoms or problems as facilitating factors (see category 9.4 in Table 3). The five participants with the top Z-change values in ICBT-i all reported acceptance of their sleeping problems in some way:

I believe I worked a lot with my thoughts about it, that I put a lot of emphasis on having to sleep a full night and get my eight hours of sleep and so on. And I got to question that quite a bit. I believe I reconsidered, a big part was that I stopped thinking so much about my sleep. It wasn't quite as important as before. I also realized I didn't need quite as many hours, that it worked well anyway, and so on. So yes, it changed quite a lot in that sense during that time. (ICBT-i, Z-change: 0.54, adherence: high)

Yes, but I guess I have accepted that perhaps… because however things are… but I have, somehow, come to terms with, with, well, my situation in life, and perhaps my sleep quality has improved thanks to that, because I have completely stopped using sleep medication, you see. Yes. In that sense I feel much better now than before. (ICBT-i, Z-change: 1.33, adherence: medium)

3.6.2. Quantitative analyses of acceptance

This finding prompted analyses of the concept of acceptance using two questionnaires that were administered in the RCT: the Sleep Problem Acceptance Questionnaire, SPAQ (Bothelius et al., 2015) and the Acceptance and Action Questionnaire II, AAQ-II (Bond et al., 2011). Both measures are based on the idea that acceptance is a mechanism of change in psychological treatments. The SPAQ is, as the name indicates, intended to measure attitudes towards sleep problems. It is originally based on data from people with insomnia, has eight items (0–6 points) and a max score of 48 points. The higher the score, the more acceptance of sleep problems. The AAQ-II is a more generic acceptance questionnaire, which is assumed to pick up changes in attitudes towards psychological problems in general, especially negative cognitions and emotions, and could thus be assumed to detect changes in acceptance also in the ICBT-d group. AAQ-II has ten items (1–7 points) and a max score of 70 points. The lower the score, the more acceptance of negative emotions and cognitions. We wanted to find out whether, as the interviews indicated, outcome was associated with changes in acceptance as measured by the questionnaires, and therefore performed correlation analyses.

Correlating the SPAQ with Z-change, we found that for ICBT-i, Z-change correlated significantly with change in SPAQ from pre- to post-treatment (r = 0.68, CI 0.28–0.88, p < 0.01). For ICBT-d there was no correlation between Z-change and change in SPAQ.

Correlating the AAQ-II with Z-change, we found that for ICBT-i, Z-change correlated significantly with change in AAQ-II from pretreatment to FU6 (r = 0.53, CI 0.02–0.82, p < 0.05). For ICBT-d there was no correlation between Z-change and change in AAQ-II.

4. Discussion

Participants in this study were diagnosed with both insomnia and depression and had received ICBT for either insomnia or depression in an RCT previously reported (Blom et al., 2015a). The aim of this study was to find factors that may have hindered or facilitated participants' work with ICBT for depression or insomnia, and use the findings as a basis to formulate hypotheses to be tested in future research. To do this we performed telephone interviews, which were the main source of information. The interview data triggered quantitative and semi-qualitative analyses intended to investigate the main themes further. The areas emerging from the interviews, leading to more in-depth analyses of other data, were opinions about the treatment, motivation and adherence, comorbidities and acceptance.

Overall, more positive statements were made about the treatment in ICBT-i than in ICBT-d, and in ICBT-d more negative statements were made than ICBT-i. This could be interpreted as indirect subjective evaluations of treatment effects, since they are in line with the results of the original effectiveness study, where ICBT-i was found more effective than ICBT-d, and led to less self-reported need for further treatment. The participants' satisfaction with the treatments, as measured by CSQ-8, was about the same as other studies using the same or very similar treatment manuals (Blom et al., 2015b, Kaldo et al., 2015a, Hedman et al., 2013) and high compared with for example an active control group in one of the trials (between group Cohen's d 0.8 Kaldo et al., 2015a). The CSQ-8 correlated with outcome for ICBT-i but not ICBT-d. Still, as many as eleven participants, a fourth of the sample, said that they would have wanted the other treatment. Only one person in the most successful subgroup, ICBT-i upper (with treatment effects above the average of ICBT-i), reported having wanted the other treatment. Similar findings were made in a qualitative study comparing CBT to psychodynamic psychotherapy, which also found that dissatisfied participants from both treatments wanted some other treatment (Nilsson et al., 2007). Prior to consenting to participation in the study, the participants were informed in several ways that the study entailed being randomized one of two treatments. Still, it is not unlikely that some participants wanted one of the treatments more than the other — unfortunately we do not have pretreatment data on any preferences they may have had. There seemed to be a qualitative difference between ICBT-i and ICBT-d in this regard — in ICBT-d the participants wanting the insomnia treatment were specific about wanting insomnia interventions. In ICBT-i, it seemed that participants wanting the depression treatment had a less specific belief, or hope, that the other treatment would have helped them past their general hindering difficulties in getting started with, and concentrating on treatment. Given the participants' wishes, and the fact that post-treatment depression rates were less than satisfactory, a natural course for future research seems to be a combination of CBT-i and CBT-d. Currently, some versions of CBT-d include limited interventions for insomnia (Hedman et al., 2013, Vernmark et al., 2010). This study and the RCT it is based on, suggest that perhaps the opposite would be more beneficial, i.e. a combined treatment with emphasis on insomnia interventions.

Hypothesis 1

A combination of CBT for insomnia and CBT for depression, with emphasis on CBT for insomnia, is more effective than only one of the treatments, for patients with comorbid insomnia and depression.

As might have been expected, more adherence to the treatment in terms of working with the treatment components, was found to be associated with a better outcome in both groups. This is in line with previous research looking into adherence to CBT (Kaldo et al., 2015b, El Alaoui et al., 2015, Geraghty et al., 2010, Matthews et al., 2013). When only looking at the number of treatment modules successfully completed, or number of days active with the treatment, this was not associated with outcome — in contrast to e.g. El Alaoui et al. (2015) who used number of completed modules as their measure of adherence and found it predictive of outcome. These somewhat inconclusive results of the adherence and outcome analyses may be due to power problems with the small subgroups in this sample, or to the fact that the participants rated their use of components after the treatment, when the result may have impacted their perception of what they did during treatment.

The two treatment groups did not differ significantly on number of comorbidities or symptom levels of depression and insomnia pretreatment. Yet participants in ICBT-d complained more that symptoms of their diagnoses and other comorbidities hindered their work with the treatment. This raises a suspicion that the depression treatment was in some way less suited for this patient group than the insomnia treatment. There are several possible explanations. One is obvious: the insomnia treatment affected symptom severity more than the depression treatment did. Hence, the participants in ICBT-d experienced more symptoms throughout the treatment and as a consequence complained more about this. Another, perhaps more speculative explanation emanating from the interviews, is that the depression treatment in itself could be more difficult to work with for patients with high symptom levels, than the insomnia treatment is. In CBT-i the main interventions are essentially about changing your bed times and spending less time in bed. CBT-d targets the very symptoms that make it difficult to adhere to treatment – low activity levels and rumination – in a way that is less precise. Lancee et al. (2014) looked at the impact of depression severity on insomnia treatment outcomes, and found that while patients with depression did benefit from insomnia treatment, more depressed patients were in more need of therapist support in order to achieve a good effect. In our study, the most successful subgroup, ICBT-i upper, had fewer comorbidities than the other subgroups, and were also the most satisfied with the support given, while the other groups wanted more and other types of support, thus corroborating Lancee's findings. It remains to be investigated what types of support are most effective, and for whom. The use of telephone support, either combined with written support or stand-alone, is worth examining further, given the input from the participants in this study. A previous study suggests that even very brief telephone support can be very effective in increasing adherence to key treatment methods (Kaldo et al., 2015b).

Hypothesis 2

Additional, or a different type of therapist support will increase outcomes for patients with more comorbidities.

We had not expected acceptance of symptom-related problems to be reported as an important facilitating factor. Accepting their sleep problems was something that stood out in the most successful ICBT-i-participants' recollections about what was helpful during treatment. No participants in ICBT-d talked of acceptance in the interviews. Interestingly, for the group receiving insomnia treatment, overall improvement did not only correlate with a measure of acceptance of sleep (SPAQ), but also with a general acceptance questionnaire focusing on negative emotions and cognitions (AAQ-II). The insomnia treatment emphasizes concrete sleep behavior changes, which are generally assumed to work via the building up of sleep pressure in combination with a stabilization of the circadian rhythm (Morin and Espie, 2003). However, sleep restriction also exposes patients to being tired and sleepy. Fearing fatigue or other daytime consequences of sleep deprivation is central to insomnia, and exposure to these feared consequences could be of great importance. This aspect is not usually emphasized in CBT-i. Whether or not acceptance is a mechanism of change in insomnia treatment – and if so, how it works – needs to be investigated further.

Hypothesis 3

Acceptance of sleep problems, and/or negative emotions and cognitions in general, is a mechanism of change in CBT-i.

There are some limitations to this study. Dividing 35 participants into four sub-groups means very low power to statistical analyses, and the statistics in this study should be viewed in that light, especially when an association one might expect, based on previous studies, was not found. The composite Z-change score used to create the sub-groups was calculated from ISI and MADRS-S measures of both the post-assessment and the six-month follow-up, to better cover the overall outcome for a patient, which means that if a participant scored very differently from one assessment to the other, this is not reflected in the Z-change score. Interpretations of the sub-group analyses must therefore be cautious. Also, all statistical analyses were induced by interview findings, not hypothesis driven. We have not statistically accounted for the risk of type I-error due to multiple comparisons, which warrants some caution. Since 32% (12 out of 37) of all the statistical tests we performed had significant results, well above the pure chance level of 5%, this is as an overall indication that the results were not just chance findings. The significant results often point in the same direction, i.e. a general superiority of ICBT-i over ICBT-d, which also indicates systematic, rather than random, effects. The interviews were conducted approximately six months after treatment, which means that it can be difficult for the interviewees to remember details, especially for those who still had symptoms of depression and insomnia. The inter-rater-reliability measure was based on recoding of only six interviews (17%). This measure should only be viewed as an indication of the face-validity of the categories generated from the interview analysis, and not as representing the final validity and reliability of the study as a whole. As with all qualitative studies, the analysts' bias will inevitably affect the analyses of the interviews and the themes chosen for further investigation.

4.1. Summary

We conclude that participants in ICBT-i were more positive about their treatment and reported fewer hindering factors for working with treatment. Having multiple comorbid conditions seems to be a hindering factor, especially for those receiving depression treatment. Adherence, defined as self-reported usage of treatment components, was positively associated with outcome. Acceptance of sleep problems, negative emotions and thoughts was positively associated with outcome in CBT-i. Proposed future research hypotheses: 1) A combination of CBT for insomnia and CBT for depression, with emphasis on CBT for insomnia, is more effective than only one of the treatments. 2) Additional, or a different type of therapist support will increase outcomes for patients with more comorbidities. 3) Acceptance of sleep problems and/or negative emotions and cognitions is a mechanism of change in CBT-i.

Declaration of interest

None.