Lonesome no more? A two-year follow-up of internet-administered cognitive behavioral therapy for loneliness

Abstract

The current study sought to investigate the long-term effects of an internet-administered programme based on CBT principles for which the initial efficacy has been reported in Käll, Jägholm, et al. (In press). Seventy-three participants who were recruited on the basis of experiencing frequent and prolonged loneliness were contacted to complete questionnaires measuring loneliness, quality of life, and symptoms of psychopathology two years after the conclusion of the initial treatment period. Additional items regarding use of the treatment techniques and strategies contained in the programme during the follow-up period was included. In total, 44 participants provided data for the loneliness measure at follow-up. The outcome data were analyzed with a piecewise mixed effects model to provide estimates of change for the continuous measures. Linear multiple regression analysis was used to investigate the relationship between use of treatment techniques and reliable change on the primary outcome measure. The results showed decreases in loneliness during the follow-up period for the sample as a whole. Additionally, an increase in quality of life and a decrease in social anxiety were noted, but no significant changes of depressive symptoms or generalized anxiety. Effect sizes for the observed changes from baseline to follow-up were in the moderate to large range for all measures. Reported use of the treatment techniques was not significantly related to reliable change in loneliness after the two-year period. In conclusion, the results of the study support the utility of internet-based CBT targeting loneliness and indicate that the benefits from the intervention can be enduring.

Highlights

• •Data was collected two years after participants received ICBT for loneliness.
• •Further reductions in loneliness compared to post-treatment were observed.
• •Effect sizes were moderate to large for all measures at follow-up.
• •Reliable change status was not related to reported use of treatment techniques.

1. Introduction

The experience of loneliness is thought to be an aversive manifestation of a perceived discrepancy between one's wanted and actual social situation (Peplau and Perlman, 1982). The reliance on the subjective perception, rather than objective characteristics, of the situation differentiates this construct from other problems such as social isolation (Wang et al., 2017). Though common and often transient, enduring loneliness has been linked to a wide range of adverse somatic consequences, including an increased risk of coronary heart disease (Valtorta et al., 2016) and all-cause mortality (Luo et al., 2012). In relation to mental health, longitudinal studies have suggested a reciprocal relationship between loneliness and symptoms of depression (Cacioppo et al., 2010) and social anxiety disorder (Lim et al., 2016). Additionally, loneliness has been linked to symptoms of generalized anxiety disorder (Beutel et al., 2017) and suicidal behavior, even when controlling for the influence of common mental disorders (Stickley and Koyanagi, 2016). Prevalence studies suggest that on average 6% of the population in many European countries experience loneliness most of the time or more often (Yang and Victor, 2011). Studies also indicate that the prevalence is substantially higher among older adults compared to the general adult population (Dahlberg et al., 2018).

Due to the adverse effects linked to loneliness and its potential relationship with symptoms of psychopathology, attempts have been made develop interventions to treat loneliness. To date, treatments based on cognitive behavioral therapy (CBT) have shown the most promise, as observed in a meta-analysis (Masi et al., 2011). Though encouraging from a treatment standpoint, the authors of the review noted that few well-designed studies were available. Since the meta-analysis, interventions employing CBT-principles have continued to show promise, though the effects have often been limited to specific populations such as older adults (Cohen-Mansfield et al., 2018; Theeke et al., 2016). In a previous study, we investigated the effects of a newly developed CBT intervention for loneliness administered via the internet where participants received continuous guidance from a therapist (Käll et al., in press-a). The programme included cognitive and behavioral techniques that sought to reduce loneliness irrespective of demographic characteristics such as age or gender. The results showed a moderate, bordering on large, between-group effect (Cohen's d = 0.77) at post-treatment on the loneliness measure when compared against a wait-list control condition that later received access to the intervention but with optional, on-demand support from a therapist. Additionally, significant differences in favor of the treatment group were found on measures of quality of life and symptoms of social anxiety. However, the long-term effects for this intervention have not yet been investigated.

The role of therapist guidance in internet-administered psychological treatments has been a topic of interest when considering the ability to disseminate such interventions broadly. While a common way of administering internet-administered treatments relies on regular guidance from a therapist (Andersson, 2016), there are also trials investigating a pure self-help format with no therapist assigned (e.g. Nordin et al., 2010). A third alternative is to provide guidance on an on-demand basis, giving the participants the option of receiving help and feedback by contacting therapist about their concerns, as seen in a study by Hadjistavropoulos et al. (2017) where participants were randomized to regular or on-demand guidance during the treatment period. In a systematic review, Baumeister et al. (2014) found that therapist support was related to better outcomes compared to unguided interventions, though the effect size for the comparison was small. Relative to on-demand guidance, studies up to this point have not found a significant difference when compared to regular guidance at post-treatment (Hadjistavropoulos et al., 2017; Rheker et al., 2015; Zetterberg et al., 2019), but this has not been investigated in the context of loneliness or over a longer period of time. Due to the lack of literature, the present study sought to explore whether this mode of support would result in improvements when analyzing the long-term outcomes.

A goal of CBT is to create lasting change. This is achieved by including treatment strategies that apply across contexts and over time, thus hypothetically ensuring that gains can be maintained beyond the acute treatment phase (Witkietvitz and Kirouac, 2016). CBT often contains an explicit focus on maintenance and so-called relapse prevention, in an attempt to prevent recurrence of psychopathology at a later time. The effects of CBT have been found to be relatively enduring in the treatment of anxiety and depression (Hollon et al., 2006), and the rate of relapse has been noted to be lower in patients treated with CBT compared to discontinued pharmacotherapy (Cuijpers et al., 2013). With regards to internet-administered CBT (ICBT), Andersson et al. (2018) found an average pre- to follow-up effect size of Hedge's g of 1.52 for interventions with at least a two-year follow-up period. While only 14 articles were included in the meta-analysis, the results suggest that the effects of ICBT can be sustained over time.

In sum, studies suggest that CBT, both in a traditional face-to-face setting and administered via the internet, can have enduring effects. However, a pure focus on outcomes gives little information about whether the maintained gains are actually related to the hypothesized elements that are designed to sustain them (e.g., changing the behavioral repertoire and promoting a focus on viable long-term goals as opposed to immediate avoidance of distress). Harvey et al. (2002) investigated whether the follow-up outcome of a CBT treatment for insomnia was related to self-reported use of the techniques and strategies contained within the intervention. The results indicated that both use of stimulus control/sleep restriction and cognitive restructuring predicted better outcomes at the one-year follow-up, but that use of relaxation techniques, which was the most frequently used component, did not improve the follow-up outcome. In a study investigating the relationship between use of treatment skills and symptom ratings, Powers et al. (2008) found a positive association between use of CBT skills and depressive symptoms two years after the participants underwent CBT. The authors hypothesized that this indicated that the participants would use the learned skills when symptoms returned. In another trial of CBT for depression, Strunk et al. (2007) found that use of CBT skills during the treatment predicted a decreased risk of relapse, though there were no retrospective data investigating use of treatment techniques during the follow-up period. With these mixed findings in mind, continued investigation of the relationship between use of techniques and outcome would seem to be of interest to help further our understanding of the effectiveness of CBT over time.

The current study serves as a follow-up to the previous investigation of the efficacy of ICBT targeting loneliness (Käll et al., in press-a) and partially make use of the data reported in that article. With regards to the limited literature on CBT interventions for loneliness and the novelty of internet-administered treatments, there is a need to investigate whether the effects of ICBT are maintained over time. Additionally, given the differences in support (fully guided for the original treatment group, on-demand support for the participants previously on the wait-list), a continued comparison between the groups would seem to be warranted. Finally, because of the lack of knowledge about the specificity of cognitive and behavioral techniques driving the decrease in loneliness, examining the association between the loneliness and use of the techniques and skills acquired in the programme could be of help for the development of future interventions. The primary aim of the present study was to investigate whether the effects were maintained over time for the treatment group. Secondly, we were interested in whether the control group that received treatment with optional on-demand therapist support had similar outcomes two years after the conclusion of the treatment phase. The third aim was to examine whether reported use of the treatment techniques and strategies were related to the likelihood of a participant achieving a reliable reduction in loneliness at the follow-up measurement.

2. Method

The current study partially make use of the data collected at pre- and post-treatment in the original study that has been reported by Käll et al. (in press-a).

2.1. Participants

Participants were recruited by means of an advertisement in a national newspaper, via social media, and posters posted at notice boards. These channels directed the participant to visit a public website with information about the study (Käll et al., in press-a). After creating an account, the participant completed a screening procedure which included demographic questions and standardized questionnaires. Inclusion criteria were: a) scoring above the reported mean (>40) on the instrument used to measure loneliness, the UCLA Loneliness Scale, version 3 (Russell, 1996), b) a reported subjective distress linked to a lasting experience of loneliness, c) an adequate ability to speak, write, and understand Swedish, and d) access to the internet via a computer or mobile device. Exclusion criteria included: a) ongoing substance abuse and b) planned changes in psychotropic medications during the initial treatment period. Psychiatric comorbidity was allowed as long the participant indicated that loneliness was their primary concern. In addition to the online screening the participants also completed a structured diagnostic interview (MINI 7.0; Lecrubier et al., 1997) by phone. The decision on inclusion/exclusion was based on the answers from the screening, the standardized diagnostic interview, and, when needed, a qualitative evaluation by the Principal Investigator. All information was hosted on a secure server and on a platform that had previously been used for similar interventions (Vlaescu et al., 2016). Details on sociodemographic characteristics can be found in Table 1. Further information about the sample and the recruitment process is available in the original article (Käll et al., in press-a).

Table 1.

Demographic characteristics of participants at baseline.

Characteristic	Treatment (n = 36)	Control (n = 37)	Total (n = 73)
Age: mean (SD)	45.6 (16.68)	48.8 (18.40)	47.2 (17.63)
Women: n (%)	26 (72.2)	26 (70.3)	52 (71.2)
Marital status: n (%)
Single Partner/married Divorced/widow/widower	16 (44.4)	19 (51.4)	34 (46.6)
	12 (33.3)	11 (29.7)	23 (31.5)
	8 (22.2)	7 (18.9)	15 (20.5)
Highest educational degree: n (%)
Primary school	1 (2.8)	1 (2.7)	2 (2.7)
Secondary school	12 (37.5)	11 (33.3)	23 (35.4)
College/university	20 (62.5)	22 (66.7)	42 (64.6)
Other vocational education	2 (5.6)	3 (8.1)	5 (6.9)
Postgraduate	1 (2.8)	0 (0)	1 (1.4)
Previous treatment for mental illness: yes n (%)	19 (52.8)	15 (40.5)	34 (46.6)
Use of psychotropic medication: n (%)
No	21 (58.3)	24 (64.9)	39 (61.6)
Yes, previously	4 (11.1)	5 (13.5)	9 (12.3)
Yes, ongoing	11 (30.6)	8 (21.6)	19 (26.1)

2.2. Procedure

The study received ethics approval from the local ethics board and began recruitment in January 2016. A total of 73 participants were included and randomized to either receive access to the treatment programme immediately (n = 36, referred to as the guided treatment group) or assigned to a wait-list with subsequent access to the programme with therapist-guidance on demand (n = 37, referred to as the guidance-on-demand group). A flowchart of the study process can be seen in Fig. 1. Post-treatment assessment was conducted directly after the initial treatment period in April 2016. The wait-listed participants received access to the treatment two weeks after this, making the total waiting period ten weeks. The treatment period for this group was identical in length to the treatment period after the initial randomization and concluded in June 2016. After the conclusion of respective treatment period, the participants did not have contact with their therapist or access to the modules. Data to investigate changes at follow-up were collected two years after the conclusion of the initial treatment period. Participants received an email with information about the follow-up and a personal link to the questionnaires. Participants that did not complete the follow-up after the first email were called and asked to fill in the forms. Additionally, two reminders were sent out via email. The outcome measures specified below was the same as used at pre- and post-treatment, while the auxiliary measures of use of treatment techniques and adverse events were unique to the follow-up time-point. A visualization of the measure points and what instruments was used at which time point can be seen in Fig. 2.

Fig. 1.

The flow of the participants through the study.

Fig. 2.

Timeline for the treatment and measurement points.

Abbreviations. ULS-3 = UCLA Loneliness Scale Version 3; BBQ = Brunnsviken Brief Quality of Life Scale; SIAS = Social Interaction Anxiety Scale; PHQ-9 = Patient Health Questionnaire 9; GAD-7 = Generalized Anxiety Disorder 7-item scale.

2.3. Treatment and therapists

The treatment was divided into eight parts referred to as modules and were administered over eight weeks. Each module contained psychoeducational text and picture elements revolving around the theme of loneliness as conceptualized in CBT terms. In addition to the text, the modules also contained assignments (homework tasks to be completed during the week). The modules were unlocked one at a time on a weekly basis and were administered regardless of whether the participant had completed all assignments in the previous module. The treatment programme was developed for the study in question and was based on behavioral and cognitive techniques such as behavioral activation and cognitive restructuring. The main focus of the intervention was to identify what might constitute valued social contact for the participant in question, increase behaviors that might realize this contact, and address obstacles that might hinder this realization (e.g. negative automatic thoughts, sensitivity to rejection). A description of the content in each module can be found in Käll et al. (in press-a). Participants in the guided treatment group were guided through the programme by a therapist, while the guidance-on-demand group had the option to contact a therapist if the need arose. This therapist provided feedback on completed assignments within 24 hours and responded to questions within the same timeframe. All therapists in the study were final year students (Masters level) from the clinical psychologist programme at Linköping University in Sweden. The guidance-on-demand group was also assigned a therapist, but the contact regarding requests for feedback and questions about the material was initiated by the participants themselves. All participants in this group received an introductory message with information about how to reach the therapist (information was also included in the initial module). Additionally, the participants received a weekly introductory message describing the module and some advice on how to approach the tasks of the week (e.g. reading through the material early on during the week to leave time for planning and completing the assignments). As with the guided group, the guidance on demand-group received access to one module per week. During the treatment period for the guidance-on-demand group, 10 of the 34 participants (29%) that remained in the condition contacted the therapist, with the frequency of contact ranging between 1 and 3 times. These 10 participants had 17 total requests for help, 12 of which were related to guidance regarding content of the modules, 3 of which were related the technical issues with the platform, and 2 that were related to the framework of the study (e.g., asking for a copy of the publications from the study).

2.4. Measures

2.4.1. Primary outcome measure

2.4.1.1. Loneliness

Loneliness was measured using the UCLA Loneliness Scale, Version 3 (ULS-3; Russell, 1996). This questionnaire consists of 20 questions that aim to capture the frequency by with the respondent experience different facets of subjective social isolation. An example item is “How often do you feel left out?”. Responses are made on a four-point scale with the alternatives “Never”, “Rarely”, “Sometimes”, and “Always”. The score on all items are summed up with a possible range from 20 (minimal/no loneliness) to 80 (maximum loneliness). The scale has been reported to have an internal consistency ranging from .89 to .94 and 1 year test-retest reliability of .73 (Russell, 1996). The internal consistency for the instrument at pretreatment in the current study was .94.

2.4.2. Secondary outcome measures

2.4.2.1. Depression

The Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001) was used to measure symptoms of major depressive disorder during the study. The instrument consists of nine items each aimed at measuring a single symptom of major depressive disorder. The score on each item, 0 to 3 points, are summed up to provide an overall indication of the presence and severity of depressive symptoms (range of 0 to 27 points). Psychometric properties for the instrument include a reported internal consistency ranging from .86 to .89 during the validation of the questionnaire. The Cronbach's α in the current sample at pre-treatment was .78.

2.4.2.2. Social anxiety

Symptoms of social anxiety were measured using the Social Interaction Anxiety Scale (SIAS; Mattick and Clarke, 1998). The instrument consists of 20 items that are answered on a five-point scale (scored from 0 to 4 points per item) which are then summed up with a possible range between 0 and 100. The internal consistency during the validation of instrument was α = .93. For the current study, the Cronbach's α at pre-treatment was found in a similar range, α = .92.

2.4.2.3. Generalized anxiety/worry

Symptoms of generalized anxiety were measured throughout the study with the Generalized Anxiety Disorder 7-item scale (GAD-7; Spitzer et al., 2006). The instrument is made up by seven items, each measuring the presence and severity of a specific symptom of generalized anxiety disorder. The score on each item is summed up to represent an overall level of symptom severity. Psychometric properties for the instrument during the validation phase include a Cronbach's α of .92 and a test-retest ICC of .83. The Cronbach's α in the current sample at pre-treatment was α = .82.

2.4.2.4. Quality of life

Brunnsviken Brief Quality of Life Questionnaire (Lindner et al., 2016) was used to measure quality of life throughout the study. The instrument contains 12 items in total, with 6 items asking about satisfaction with different areas of life. Each item is paired with a follow-up question regarding the importance of the area to the respondent's quality of life. Psychometric properties for the instrument during the validation phase include a test-retest ICC of .86 and an internal consistency of Cronbach's α = .76. The correlation with the Quality of Life Index (QOLI) was reported as .65 during the validation of the instrument (Lindner et al., 2016). In the same study, the negative convergent validity with PHQ-9 was noted at r = −0.51, significantly lower than the coefficient for QOLI. The product of each item pair is summed up to give a measure of overall quality of life. The internal consistency for the current sample at baseline was Cronbach's α = .82.

2.4.3. Other measures

2.4.3.1. Adverse events, changes in medication, and additional psychotherapy

Four additional questions were used to investigate the impact of factors that were deemed as plausibly relevant to the experience of loneliness after the follow-up period. Participants were asked to indicate whether they had experienced any adverse life events, had undergone any additional psychotherapeutic treatments, had started taking or changed their dosage of a psychotropic medication, or had been diagnosed with a somatic illness or physical disability. The questions about additional psychotherapy and changes in medication had been asked at the post-treatment assessment with a different phrasing to represent the difference in timeframe (eight weeks versus two years), but the other items were administered for the first time. Participants responded on dichotomous scale, no/yes (coded as 0 or 1).

2.4.3.2. Use of treatment techniques during the follow-up period

To investigate the relationship between the reported use of treatment techniques included in the programme and the primary outcome, an additional set of questions was created and administered. The questions were only administered during the follow-up assessment when the participants were asked to recall their use of treatment techniques since the end of the treatment. The techniques in question were: use of functional analysis, using goal-setting strategies and exploring personal values, cognitive restructuring techniques, strategies to end rumination, behavioral experiments, behavioral activation to increase the amount of valued social contact, and exposure with a reduction of safety behaviors. Each question included the name of the technique (e.g. exposure) and a brief description of what it entails (e.g. “to systematically and deliberately approach situation that evoke anxiety without the use of safety behaviors that reduces the anxiety short-term”). The participants were asked to indicate how often they had been using each technique during the follow-up period using one of four alternatives: not at all (scored as 0), on a few occasions (scored as 1), sometimes (scored as 2), and regularly (scored as 3). The items were not based on any existing questionnaire, but rather aimed to capture the techniques learned throughout the specific modules in this study. Two principal scorings were set up a priori: firstly, total use of techniques calculated by summing up the score on each item (possible range: 0–28). Secondly, active use of techniques operationalized as the total number of techniques which the participant indicated that they used sometimes or regularly (i.e. a score ≥ 2 on the individual items; possible range: 0–7).

2.5. Statistical analysis

All analyses were conducted using R version 2.6 (R Core Team, 2019) or IBM SPSS version 25. Across analyses the alpha level was set to p < .05 and confidence intervals are reported at 95%.

2.5.1. Primary and secondary aims

For the primary and secondary aim, the following analyses were used: Comparisons between those who completed the measurement and those who did not at follow-up were made using χ²-tests and independent t-tests. The main analytical model employed was a conditional piecewise mixed-effects model used for estimating change over time across and between the two conditions over time with the nlme package (Pinheiro, Bates, DebRoy, Sarkar, and R Core Team, 2018). Fitting a piecewise model to the data allows for estimation of change within distinct phases of a longitudinal trial (Raudenbush and Bryk, 2002). For the present study the treatment period (pre-treatment to post-treatment) and the follow-up period (post-treatment to 2-year follow-up) were modeled as such phases to investigate change on the outcome measures during the follow-up period. The fixed effects of interest were the effect of time (i.e. change from post-treatment to follow-up), group, and whether there existed differences between to conditions (coded as guidance-on-demand = 0, original treatment group = 1) for this period with a Time X Group interaction. The model was estimated using a diagonal covariance structure. Due to lack of measurement points, random slopes were not estimated for any of timepieces and the covariance between the slopes and the intercept were fixed at 0. Significance of the fixed effects were tested using a Wald-test (the estimate divided by the standard error compared against a z-distribution) with a two-tailed significance level of p < .05 (corresponding to a z-score of ±1.96). The intention-to-treat (ITT) principle were used for the outcome measures, meaning that all randomized participants were included in the analysis regardless of whether they provided data at the post- and/or follow-up measurement. Missing data were handled using restricted information maximum likelihood (REML). This method of handling missing data provides unbiased estimates under the assumption that data is Missing At Random (MAR), which allows data to be missing as a function of the value of other observed variables but not the would-be value on the measure (Enders, 2010). This assumption is less strict compared to Missing Completely at Random (MCAR) where missingness is not allowed to be related to any observed variable. Standardized effect sizes for the between- and within-group comparisons were calculated using the observed mean difference divided by the pooled standard deviation, also known as Cohen's d. These were interpreted in accordance with Cohen's (1988) rule of thumb meaning that 0.30, 0.50, and 0.80 correspond to a small, moderate, and large effect size respectively. Reliable change was calculated using Jacobson and Truax's (1991) formula where the mean from the baseline measurement is subtracted from the observed mean at follow-up and then divided by the standard error of measurement of the difference for the primary outcome measure. For the current study, a change value of ±8.71 represented the cut-off for reliable change/deterioration. A multiple binary logistic regression analysis employing forced entry was used to investigate the relationship between other external factors (e.g. adverse events) and reliable change.

2.5.2. Tertiary aim

For the tertiary aim, the relationship between use of psychotherapeutic techniques and reliable change was investigated using a multiple linear regression analysis with the use of each techniques entered as a predictor using forced entry for the outcome (i.e. reliable change at follow-up, dummy-coded as 0 = no reliable change, 1 = reliable change at follow-up).

3. Results

3.1. Attrition, missing data, and completion of modules

Missing data at the follow-up measurement ranged between 44% (BBQ, GAD-7, PHQ-9) and 41% (UCLA-LS-3). There was no significant difference between conditions in terms of response rate, χ²(1) = .66, p = .417. Comparisons between completers and non-completers of the follow-up measurement did not indicate any significant differences at follow-up or the earlier measurement points with regards to outcome measures (all p > .10) or demographic variables at pre-treatment (all p > .13).

In the guided treatment condition, participants accessed on average 5.88 modules (SD = 3.03) and completed (as in completed all assignments in a module to a satisfactory degree) on average 4.89 (SD = 3.03) of the 8 modules. In the guidance-on-demand condition, participants accessed on average 4.47 (SD = 3.22) modules and completed on average 3.41 (SD = 3.11) of them.

3.2. Change on the outcome measures from post-treatment to the two-year follow-up

Observed means for the outcome measures are detailed in Table 2. Effect sizes for the observed differences within and between groups can be found in Table 3.

Table 2.

Observed means, standard deviations, and sample sizes for the outcome measures.

Measure	Pre-treatment	Post-treatment	Two-year follow-up
	M (SD)	M (SD)	M (SD)
ULS-3 (loneliness)
Treatment	58.61 (4.15), n = 36	50.52 (6.95), n = 27	47.57 (8.48), n = 21
WL/on demand	59.62 (7.47), n = 37	56.24 (9.41), n = 34	48.23 (12.99), n = 22
BBQ (quality of life)
Treatment	32.61 (17.21), n = 36	45.48 (16.95), n = 27	47.26 (20.05), n = 19
WL/on demand	32.14 (17.86), n = 37	32.06 (18.67), n = 34	52.45 (23.78), n = 22
SIAS (social anxiety)
Treatment	31.81(14.12), n = 36	25.41 (12.20), n = 27	23.60 (11.05), n = 20
WL/on demand	34.39 (16.90), n = 37	31.76 (16.40), n = 34	25.77 (17.09), n = 22
PHQ-9 (depression)
Treatment	10.14 (5.68), n = 36	6.26 (4.10), n = 27	6.05 (5.17), n = 19
WL/on demand	9.46 (4.30), n = 37	8.09 (4.83), n = 34	6.32 (6.11), n = 22
GAD-7 (generalized anxiety)
Treatment	7.03 (4.60), n = 36	4.89 (3.64), n = 27	4.21 (4.21), n = 19
WL/on demand	6.76 (4.40), n = 37	6.35 (4.47), n = 34	3.68 (3.00), n = 22

Abbreviations. ULS-3 = UCLA Loneliness Scale, Version 3; BBQ = Brunnsviken Brief Quality of Life Scale; SIAS = Social Interaction Anxiety Scale; PHQ-9 = Patient Health Questionnaire 9; GAD-7 = Generalized Anxiety Disorder 7-item scale.

Table 3.

Effect sizes (Cohen's d) with 95% CIs within and between conditions for the pre- to two-year follow-up comparison using the observed Means.

Outcome measure	ESWithina	ESBetweenb
ULS-3		0.06 [−0.56, 0.68]
Treatment	1.65 [1.02, 2.29]
WL/on demand	1.07 [0.5, 1.65]
BBQ		−0.24 [−0.87, 0.4]
Treatment	0.78 [0.2, 1.37]
WL/on demand	0.97 [0.4, 1.53]
SIAS		0.15 [−0.47, 0.78]
Treatment	0.65 [0.08, 1.22]
WL/on demand	0.51 [−0.04, 1.05]
PHQ-9		0.05 [−0.59, 0.68]
Treatment	0.75 [0.17, 1.34]
WL/on demand	0.59 [0.04, 1.13]
GAD-7		−0.15 [−0.78, 0.49]
Treatment	0.64 [0.06, 1.22]
WL/on demand	0.82 [0.25, 1.36]

Abbreviations. ULS-3 = UCLA Loneliness Scale Version 3; BBQ = Brunnsviken Brief Quality of Life Scale; SIAS = Social Interaction Anxiety Scale; PHQ-9 = Patient Health Questionnaire 9; GAD-7 = Generalized Anxiety Disorder 7-item scale.

^aES_within. = effect size within each condition from pre-treatment to follow-up.

^bES_between = effect size between the original treatment group and the guidance-on-demand group at follow up.

3.2.1. Change in loneliness

The piecewise mixed model indicated a significant reduction of loneliness for the entire sample during the follow-up period, b = −7.96 [−11.70, −4.23], SE = 1.88, p = .0001. The reduction was significantly steeper in the group that received treatment with guidance-on-demand during the follow-up period after a waiting period, b = 5.77 [0.32, 11.23], SE = 2.75, p = .0382.

3.2.2. Change in quality of life

For the quality of life measure, a significant increase was found at follow-up for the whole sample, b = 18.78 [11.42, 26.14], SE = 3.71, p > .0001. The increase at follow-up was larger in the guidance on-demand group than in the original treatment group, b = −17.93 [−28.95, −6.91], SE = 5.56, p = .0017.

3.2.3. Change in social anxiety

Across the sample, the estimates for SIAS showed a decrease during the follow-up period, b = −5.74 [−9.93, −1.55], SE = 2.11, p = .0078. The Time X Group interaction was not significant, b = 4.70 [−1.54, 10.93], SE = 3.14, p = .14.

3.2.4. Change in depression

For the PHQ-9 measure, there was no significant change from post-treatment, b = −1.98 [−4.27, 0.32], SE = 1.16, p = .09. The difference between the two groups during the period was not significant, b = 1.82 [−1.60, 5.26], SE = 1.73, p = .29.

3.2.5. Change in generalized anxiety

The results from the model indicated a significant change on GAD-7 at follow-up for the sample, b = −2.46 [−4.26, −0.66], SE = 0.91, p = .0078. The difference in change between the two conditions during this phase was not significant, b = 2.08 [−0.61, 4.77], SE = 1.36, p = .13.

3.3. Reliable change of loneliness at the two-year follow-up

Of the 44 (60%) participants who completed the follow-up measurement, 59.1% (n = 22) had a loneliness score that indicated reliable change from pre-treatment to follow-up as per Jacobson and Truax (1991) definition. One participant (2.3%) was classified as reliably deteriorated.

3.4. Reported use of psychotherapeutic techniques during follow-up period

Descriptive statistics for reported use of treatment techniques and unstandardized beta coefficients for the relationship with reliable change can be found in Table 4. The reported total use of techniques for the sample was 8.75 (SD = 5.70) out of a maximum of 21. On average, participants reported having used 2.84 techniques (SD = 2.46) actively during the follow-up period.

Table 4.

Reported use of psychotherapeutic techniques during the follow-up period and the relationship between use of techniques and reliable change during the study.

Technique	Reported use (SD)	b	SE	p-Value
Functional analysis	1.09 (0.96)	.12	.11	.30
Goal-setting	1.43 (1.09)	.14	.10	.17
Cognitive restructuring	1.68 (1.10)	.00	.11	.99
Strategies for rumination	1.39 (1.17)	.08	.09	.38
Behavioral experiments	0.80 (0.93)	−.20	.11	.07
Behavioral activation	1.20 (1.00)	.11	.12	.35
Exposure	1.16 (1.06)	−.11	.09	.22

3.5. Events, illness, additional psychotherapy, and changes in medication

No between-group differences were reported for the use of other psychological treatments, changes in psychopharmaceutic medications, or incidence of adverse events during the follow-up period (all p > .52). The original treatment group did report a higher incidence of somatic illness and/or physical disability during this period, χ² = 5.25, p = .022. Beta coefficients and odds ratios for the external factors of interest can be found in Table 5. Of the 46 participants who responded to the question, 41.3% (n = 19) reported having experienced one or more adverse events during the follow-up period. A total of 14 participants (30% of the responding participants) reported having started or changed their dosage of psychotropic medication during the follow-up period. In total, 14 of the respondents (30%) replied that they had received additional psychological interventions after the post-treatment assessment. Lastly, 28.3% (n = 13) indicated that they had experienced the incidence of a somatic disease or physical disability during the follow-up period.

Table 5.

Beta estimates for the relationship between adverse events, changes in psychotropic medication, somatic disorder/sickness, and additional psychotherapeutic treatment during the follow-up period and reliable change during the study.

Factor	b	SE	p-Value	Odds ratio (95% CI)
Adverse event(s)	−.61	.75	.42	0.55 (0.12–2.36)
Changes in psychotropic medication	−.14	.89	.87	0.86 (0.15–4.95)
Sickness/disability	.21	.81	.79	1.24 (0.25–6.10)
Psychotherapeutic treatment	1.41	.78	.07	4.08 (0.89–18.72)

4. Discussion

The aim of the study was to investigate the long-term effects of an ICBT intervention targeting the experience of loneliness. Additionally, the collected data sought to investigate long-term changes on symptoms of psychopathology and quality of life, as well as the relationship between reliable change on the primary outcome measure and the reported use of techniques and strategies presented in the programme.

4.1. Changes in the primary and secondary outcomes at the two-year follow-up

Overall, the results indicated a significant decline on loneliness from the post-treatment measurement to the follow-up for the entire sample. The significant interaction between time and condition suggests that the reduction was steeper in the guidance-on-demand group. This significantly steeper decline in loneliness in this group is consistent with the fact that the follow-up measurement served as the measurement point after the group received their version of the treatment. Due to the lack of an earlier measurement where both groups had received their version of the treatment, we expected a larger decline in the guidance-on-demand group under the assumption that this version of the intervention was at least roughly as efficacious as the one that the original treatment group received. The estimates and the observed means in both groups indicate an enduring reduction of loneliness. However, the lack of a control group that has not been exposed to the intervention at the follow-up measurement means that the data do not allow for causal inferences about the long-term impact of the programme itself. The within-group effect sizes for pre- to follow-up comparison in both groups from baseline to follow-up were large (1.65 and 1.06 for the group with therapist guidance and guidance-on-demand, respectively). The differences in effect size between the groups would seem to be due to the consistently larger standard deviations in the group that received guidance-on-demand, as inferred by the fact that the observed and estimated means at follow-up are very similar. Due to how the within-group effect size is calculated (the mean difference between the pre- and follow-up measurement divided with the pooled standard deviation), differences in the standard deviations could cause differences in the within-group effects sizes, yet only trivial differences between the groups (because of the fact that the calculation of this effects size pools the standard deviations from both conditions).

Fifty-nine percent of the participants who responded to the follow-up were classified as having achieved a reliable change. Arguably, the reduction of loneliness in this study is greater than that observed in some recent studies on treatments with CBT-elements for loneliness (Cohen-Mansfield et al., 2018; Theeke et al., 2016), but also smaller than an earlier intervention employing chat-based CBT (Hopps et al., 2003). Compared to recent studies, the observed within-group effects are lower than that of a trial investigating mindfulness techniques (Lindsay et al., 2019) but higher than the one found in a recent trial investigating use of a intervention built around the concept of social group memberships (Haslam et al., 2019). For these studies however, it should be noted that the time frame and instruments used for measuring loneliness differs from the design of the present study, thus making direct comparisons difficult. Compared to the long-term effects of other ICBT interventions summarized by Andersson et al. (2018) the effects sizes in the present study were in the lower range. In sum, while treatment effects were maintained over time, they appear to be lower than the average effect from comparable internet interventions for other conditions.

The results on quality of life and symptoms of social anxiety indicated a higher quality of life and lowered symptoms of social anxiety after the follow-up phase. For the quality of life measurement, this change was primarily driven by the higher ratings in the guidance-on-demand group. Though not significant for the symptoms of social anxiety, the interaction between time and group showed a trend in the same direction. The within-group changes from baseline to follow-up for these measures were in the large (for quality of life) to moderate (for symptoms of social anxiety) range. The effect sizes for these measures are comparable to other ICBT studies providing data on the long-term effects on secondary outcomes (El Alaoui et al., 2015; Rozental et al., 2017), and suggest that the positive impact of the intervention was not restricted to the participants' experience of loneliness.

For the measures of depressive symptoms and symptoms of generalized anxiety, the reductions during the follow-up phase were not statistically significant. Both of these were considered secondary measures and was not found to change significantly during the original treatment period either (Käll et al., in press-a). The observed mean at follow-up for the GAD-7 indicate a minimal symptom level (Spitzer et al., 2006), while the level of depressive symptoms on average was in the mild range (Kroenke et al., 2001).

The marginally significant relationship between additional psychotherapy and reliable change during the study (while accounting for the other factors) is interesting. The fact that participants who had received additional psychotherapy during the two years since the end of the original study was more likely, although not significantly so, to have undergone reliable change points to the need to assess additional treatments carefully when investigating long-term effects of interventions. This marginal finding could imply that it was this additional psychotherapy that can be seen as the driving factor behind the observed changes. However, the item provided little information about what interventions the participants had received and whether there are similarities between the interventions in question and the one in this study.

4.2. Differences between modes of support

The between-group difference on the primary outcome measure was very small, indicating that there was no meaningful difference between the guided and the guidance-on-demand group in loneliness at follow-up. Furthermore, only 29% of the participants in the guidance-on-demand group requested guidance, and none had more than three requests for contact with the therapist. The similarities in observed means is consistent with earlier studies that have not found significant differences between these of modes of support (e.g. Hadjistavropoulos et al., 2017; Rheker et al., 2015; Zetterberg et al., 2019). These results could suggest that a high level of therapist-support (i.e. continuous, structured support every week) is not needed for internet interventions targeting loneliness. The previous literature on the role of therapist support in internet-delivered psychological treatments is somewhat inconsistent as, for example, a review of internet-administered interventions for depression (Johansson and Andersson, 2012) found that more frequent and structured therapist-support was associated with larger effect sizes. While the results in the present study does not point at any significant differences at follow-up, the study design and the sample size prevents us from drawing any conclusions about the relative efficacy of therapist-support when treating loneliness. The data do not give information about possible differences in rate of change nor whether there exist differences in when the change occurred between the groups. The role of therapist-support would seem to be of interest given the link between loneliness and the perception of social support, and further research could investigate whether therapist-support is needed and/or preferable in this population when delivering ICBT. As of now, our recommendation is that regular therapist-support is included in similar interventions until the comparison with guidance-on-demand has been tested in a sample with greater statistical power to detect differences.

4.3. Relationship between use of techniques and reliable change

With regards to the third purpose of the study, the reported use of treatment techniques was not very frequent in the sample as a whole. Even the most frequently used technique, cognitive restructuring, only had a mean that indicated use on a few occasions/sometimes during the follow-up period. Furthermore, none of the techniques exhibited a significant relationship with reliable change during the study. This can be contrasted against findings from other studies such as Harvey et al. (2002) in which the authors found a significant relationship between use of techniques that was theoretically specified as key elements in the treatment and the follow-up outcome. There are several possible factors that might have contributed to the different results in the present study. First, there is currently little knowledge about what constitutes a key element in interventions targeting loneliness. Though efforts have been made to explore possible key elements (e.g. Käll et al., in press-b), the mechanisms of change in this category of interventions remain to be explored. Second, the long period between the measurement points is problematic with regards to the participants' recall of the principles and how often they have been used during the follow-up period. Using more frequent measurement points would help alleviate this problem while also providing the ability to investigate this relationship over time in greater detail (e.g., directionality). Third, the questions and response alternatives used to measure the association had not been validated prior to their use in this study. Future studies might also benefit from inquiring into more detailed aspects of the use of techniques. For example, infrequent use could point to a lack of recall of the strategy, but also to the fact that the person might not have been confronted by the situation that the technique aims to help resolve. Investigations into this relationship might stand to gain by collecting data about why and how the participants use the technique in question. Bearing these points in mind, although the results do not suggest a significant relationship, further investigations are warranted to help clarify if and how the rehearsal and use of the techniques actually help achieve enduring results in ICBT.

4.4. Limitations

There are limitations of the present study. The response rate at the two-year follow-up was rather low with only 59% of the participants providing data on the primary outcome measure. This is lower than most of the follow-up studies in the area, which often have a response rate around 80% (Andersson et al., 2018). Our estimates were derived under the assumption that the data were missing at random (i.e., that there was no systematic pattern of missingness with regards to the would-be values). Although there were no statistically significant differences between responders and non-responders on any of the measurement points, this assumption is not directly testable. The low number of participants is also problematic in relation to the second aim of the study. Given the earlier literature (e.g. Hadjistavropoulos et al., 2017), we did not expect to find a substantial difference between the conditions, if a difference at all. Given this, the study was underpowered for this comparison and the results should be viewed with this in mind. Furthermore, the number of participants in study as a whole and the number who completed the follow-up assessment in particular were too low to provide reliable estimates for the predictors of reliable change. In sum, the large percentage of participants who did not complete the follow-up assessment is troublesome and the conclusions of the study should be interpreted with caution.

Another limitation previously mentioned briefly is the lack of measurement points during the follow-up period. This prevents us from drawing more detailed conclusions about the nature of the change (i.e. when it occurred and whether there existed differences between the groups in the rate of change). The ability to investigate the relationship between the use of treatment techniques and outcomes is also affected by this. The fact that the follow-up measurement served as the first time point after the post-treatment assessment for the guidance on demand-group also limits the ability to access the impact of the ICBT component and does not allow us to differentiate this from what might be considered the natural trajectory of loneliness over time. Future studies would benefit from including additional measurement points during the follow-up period.

Of relevance when considering the tertiary aim of the study is the possible limitations of the instrument administered to measure use of treatment techniques. The instrument was created for this specific measurement point and had thus not been tested previously. An additional issue to consider is the time scale used. The answering alternatives used could be prone to provide flawed estimates that might both over- and underestimate the extent by which the participant made of the strategies. For example, someone using the techniques daily for the first few months of the follow-up period, but not at all during the second half might rate the frequency as using them as sometimes, even though the participant in question made more frequent use of them then someone used the technique once or twice every month yet also gave the same response. A possible remedy to this could be to a) have continuous ratings during the time frame of interest and b) provide answering alternatives that are more standardized and easier for the participant to interpret (daily, weekly, a couple of times a month etc.). The way the questions were constructed combined with the inherit possibility of a flawed recall of how and when techniques were used during this long of a follow-up period make it so that the results should be viewed as exploratory.

Finally, the use of inclusion and exclusion criteria and the means by which the participants were recruited may inadvertently have served to homogenize the sample. This might in turn have an impact on the investigation of the predictors in the study. As Titov et al. (2010) noted, samples in ICBT studies have been fairly homogeneous with regards to demographic characteristics. This is a threat to the external validity of the findings, but also a potential problem when investigating predictors of change in this field. The sample in the present study does have commonalities with the typical characteristics for ICBT trial (e.g. higher level of education, a higher percentage of females than what would be expected by pure chance), and this may have served to muddle the detection of significant predictors of change.

5. Conclusions

In conclusion, the results suggest that the effects of ICBT targeting loneliness can be enduring. The benefits from the intervention does not seem to be limited to a reduction in the frequency by which the participants experience loneliness, but also an increased quality of life and reduced symptoms of psychopathology. No significant relationship was found between response status and reported use of techniques during the follow-up period. Though preliminary at this point, the results can be viewed as support for CBT-interventions targeting loneliness in general, and more specifically the possibility of alleviating loneliness by means of internet interventions. Future studies should seek to test the effectiveness of this and similar interventions in regular care seating, potentially both at as a stand-alone primary care intervention but also as an adjunct intervention at a psychiatric level given the noted comorbidity with psychopathological symptoms.

Declaration of competing interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

Appendix A. Instrument used to measure use of techniques during the follow-up period

Below is a list of the techniques and strategies that was included in the modules. Pick the alternative that best describes how often you have used these since the end of the treatment period.

		Not at all	On a few occasions	Sometimes	Regularly
Analyzing behavior using vicious/virtuous circles	Example: looking at what thoughts, feelings and behaviors exists in a situation and the short- and long-term consequences.
Using goals and values	Example: breaking down long-term goals into smaller parts, thinking about what you value in a certain area of your life.
Investigating negative automatic thoughts and avoiding thought traps	Example: registering thoughts and rating the truthfulness of them.
Breaking out of rumination	Example: attempt to stop comforting thoughts that keep the rumination going.
Behavioral experiments	Example: test out your assumptions and predictions and compare against actual results.
Behavioral activation/increasing your amount of valued social contact	Example: planning for social actions, identifying potential obstacles.
Exposure	Example: to systematically and deliberately approach situation that evoke anxiety without the use of safety behaviors that reduces the anxiety short-term