Acceptability, satisfaction and perceived efficacy of “Space from Depression” an internet-delivered treatment for depression

Abstract

Background

There are clear advantages to internet-delivered interventions for depression. Users' perspectives on the acceptability, satisfaction, and efficacy of an internet-delivered treatment for depression can inform future developments in the area.

Methods

Respondents (n = 281) were participants in an 8 week supported internet-delivered Cognitive Behaviour Therapy treatment for depressive symptoms. Self-report online questionnaires gathered quantitative and qualitative data on the user experience.

Principle findings

Most respondents were satisfied with the programme (n = 191), felt supported (n = 203), reported positive gains and impact resulting from use of the programme, and perceived these to be likely to be lasting effects (n = 149). Flexibility and accessibility were the most liked aspects. A small number of respondents felt their needs were not met by the intervention (n = 64); for this group suggestions for improvements centred on the programme's structure and how supporter feedback is delivered.

Conclusion

Results will deepen the understanding of users' experience and inform the development and implementation of evidence-based internet-delivered interventions.

Highlights

• •Similar to previous research on internet-delivery the majority of respondents reported satisfaction (68%) with their intervention.
• •Respondents reported efficacy and empowerment and domains including key CBT techniques, strategies and lifestyle changes.
• •Similar to previous research, flexibility and accessibility help to overcome barriers, accommodate individual needs and increase autonomy.
• •Digital access and computer literacy are barriers for underrepresented groups including older adults and those of low socio-economic status.
• •Support is important, but more tailored feedback may be helpful. Implications include supporter training and governance issues.

1. Introduction

1.1. Background

“Depression is a leading cause of disability worldwide, and is a major contributor to the global burden of disease” (WHO, 2012). For instance, approximately 1 in 4 individuals experience mental disorders across Europe in their lifetime (Alonso et al., 2004, Ayuso-Mateos et al., 2001); with the highest prevalence of depressive disorders reported in urban UK (17%) and Ireland (12.3%). The majority of people with depression also present with significant functional impairment in their personal, social and occupational life (Kessler et al., 2003, Rapaport et al., 2005). Functional impairment is a major cause of distress for those living with depression.

The World Health Organisation (WHO) has highlighted depressive disorders as one of the most costly disorders internationally in relation to healthcare usage and disability (Richards, 2011). The national depression charity AWARE reported that over 300,000 individuals experience depression at any one point in time in Ireland, and that approximately 1 in every 14 employees are affected (AWARE 2009 as cited in Department of Social Protection, 2013). One estimate of the economic cost of depression, related to occupational functioning alone, is 280 million euro per year (Department of Social Protection, 2013).

1.2. Access to evidence-based treatments

Depression is clearly established as a serious public health concern and can be treated relatively successfully using antidepressants, but relapse is high following cessation, and many patients prefer psychological therapies (Van Schaik et al., 2004). The National Institute for Health and Clinical Excellence (NICE, 2009) guidelines outline that individuals living with depression should have access to evidence-based psychological interventions such as Cognitive Behaviour Therapy (CBT), which has been established as an effective treatment of depressive disorders. The US based National Institute of Mental Health (NIMH) Psychosocial Intervention Development Workgroup also provide similar recommendations on treatments for individuals experiencing depression (Hollon et al., 2002).

In Ireland, expert reviews have highlighted the need for alternatives to pharmacotherapy and therefore access to evidence-based psychological interventions as an integral part of care (Department of Health and Children, 2006). However, a gap in availability of these services for those seeking help with their mental health difficulties remains. In some other cases individuals may encounter barriers to help seeking or accessing interventions such as situational, financial, perceived lack of effectiveness and stigma (Kessler et al., 2001). Further, negative perceptions of interventions may be a factor in high attrition rates and failure to seek help, highlighting the need to understand user perceptions and incorporate their feedback.

1.3. Online interventions for depression & the user-experience

In recent years interventions for depression have been delivered online to users (Cuijpers and Riper, 2015). Internet-delivered interventions supersede computer based interventions that included CD-ROM delivery, but also more recently online delivered. Internet-delivered interventions are entirely delivered through the internet and have the potential to provide a person-centred environment where the user takes control and actively participates in the management of their care. This delivery modality promotes anonymity, expression, reflection and empowerment while creating a sense of achievement and recording individual progress (Wright, 2002). There are clear advantages to the use of Internet-delivered interventions and one focus of current research is on providing evidence for the acceptance, efficacy and satisfaction with these methods.

Computerised Cognitive Behaviour Therapy (cCBT) has been recommended as a structured alternative to traditional low-intensity methods (NICE, 2006). Evidence suggests that it is an acceptable and effective format of delivery, demonstrating significant clinical outcomes and levels of satisfaction. Studies of cCBT have demonstrated the importance of therapist support in driving user engagement and improving overall experience (Richards and Richardson, 2012).

Kaltenthaler et al. (2008) carried out a systematic review of user acceptability and satisfaction within cCBT research. They found CCBT to have an overall positive response across studies with participants reporting satisfaction and ease of use of the intervention as well as accessibility and positive impact resulting in improved quality of life. Factors affecting user acceptability included personal motivation, mode of delivery and perceived benefits or demands. A limitation identified was the inclusion of only those who had completed the treatment, which did not provide information as to why people did not engage with the online intervention. The need for further research on users experience of cCBT interventions was highlighted.

In their comparison study of email-delivered CBT versus cCBT, Richards and Timulak (2013) investigated users satisfaction and the helpful aspects of these treatments. The majority of respondents found cCBT helpful, easy to use and effective with no significant differences across groups. Positive elements of the interventions included a sense of self-control and anonymity, CBT techniques and engaging, user-friendly, content. What users reported liking the least referred to finding the programme and content demanding, complicated, impersonal and not meeting their individual needs.

In a review of cCBT treatments for depression a number of advantages and disadvantages of this type of intervention were discussed (Eells et al., 2014). Key aspects in delivering an effective online intervention include clinical training, informing users of its evidence base and the use of some form of therapist support.

There are various existing cCBT interventions tailored to meet the needs of users. Research on users' feedback has identified key features for improving engagement and overall satisfaction with online interventions such as the integration of an online supporter, use of evidence-based techniques, self-administration, anonymity and engaging and user-friendly content (Eells et al., 2014, Richards and Timulak, 2013).

The ‘Space from Depression’ programme has incorporated these key features into an internet-delivered cognitive behavioural therapy (iCBT) intervention. Weekly reviews by a trained supporter provide guidance, feedback and motivation for the user. Psycho-educational content is delivered in multiple formats to facilitate acquisition of knowledge and promote usability. Interactive tools and activities aim to reinforce learning and to encourage reflection and implementation of new skills. The user is provided with access to a non-linear modular programme, with the objective of creating a sense of autonomy and anonymity.

In order to continuously inform and improve psychological interventions, the understanding and acknowledgement of the users and clients experiences as experts in their own care is crucial. It is also important to understand users experiences, perceptions and satisfaction with interventions as these factors have been shown to be linked to improved functioning, clinical outcomes, and improved attrition rates (Ankuta and Abeles, 1993).

1.4. Objectives

The purpose of the current study was to gain insight into users' experiences of a supported internet-delivered low-intensity treatment ‘Space from Depression’, for symptoms of depression within an Irish adult community population.

2. Methods

2.1. Design

This study was part of a large-scale randomised control trial (RCT) that principally examined the effectiveness of an online intervention for depression, Space from Depression, in the community. The study employed a mixed methods approach, and also sought to examine users' experiences (acceptance and satisfaction) with the internet-delivered treatment for symptoms of depression.

2.2. Recruitment

Participants were recruited through self-referral from an adult community sample from the Aware Charity, a national depression charity in Ireland. Participants obtained information about the study, what was involved in participating, the treatment, and how to make contact and proceed with screening from the Aware website. On agreeing to participate, informed consent was completed online and thereafter baseline screening assessments (demographic and clinical characteristics, BDI-II, GAD-7, Work and Social Adjustment Scale). The protocol for the trial is described elsewhere (Richards et al., 2014).

2.3. Participants

A total of 641 participants were recruited of which 281 respondents were included in the current analysis (N = 281). To provide a comprehensive overview of experience, respondents included in the results were those who registered with the programme, provided socio-demographic details and answered at least one question related to the user experience and satisfaction, which was administered post-treatment. Participants were excluded where they had signed up for the programme but had not completed any modules or provided feedback on the user-experience questions. The characteristics of the sample are presented in Table 1.

Table 1.

Socio-demographic characteristics of the sample.

Socio-demographics	N = 281	% sample
Gender
Male	70	24.91
Female	211	75.09
Age
Range	18–63
Mean	38.10
Education level
High school	51	18.15
Undergraduate degree	93	33.10
Postgraduate degree	65	23.13
Other certificate	66	23.49
None	6	2.14
Confidence using computers and internet
Very confident	145	51.60
Confident	88	31.32
Average	42	14.95
Mildly confident	6	2.14
Not confident	0	0
Employment
Part-time or student	74	26.33
Fulltime	122	43.42
Unemployed	37	13.17
Retired	8	2.85
Disabled	3	1.07
At home parent	37	13.17
BDI (Beck Depression Inventory)
Sub-clinical	51	18.15
Mild	61	21.71
Moderate	93	33.10
Severe	76	27.05
Previous treatment for depression
Did not answer	9	8
No	86	31
Yes*	168	60
Medication	40	14
Counselling/psychotherapy	34	12
Medication and counselling	93	33

Beck Depression Inventory (BDI-II) levels of severity; minimal (0–13); mild (14–19); moderate (20–28); severe (29–63); * = n = 10 did not report on this.

2.4. Intervention

2.4.1. Computerised Cognitive-Behaviour Therapy (cCBT) programme

The online intervention was ‘Space from Depression’, an eight-module online CBT-based intervention for depression, delivered on a Web 2.0 platform using media-rich interactive content. Programme content is delivered in a non-linear fashion. Each module takes roughly 1 h to complete and it is recommended one module be completed per week. The structure and content of the programme modules follow evidence-based CBT principles. The treatment comprises cognitive and behavioural components including self-monitoring and thought recording, behavioural activation, cognitive restructuring, and challenging core beliefs. Each module is structured in an identical way and incorporates introductory quizzes, videos, informational content, interactive activities, as well as homework suggestions and summaries. In addition, personal stories and accounts from other users are incorporated into the presentation of the material. A description of the programmes eight modules is provided in Table 2.

Table 2.

Outline of programme modules.

Modules	Description
SilverCloud	This a technical module that introduces the user to the platform. Describing core platform features; icons, layout, tools and activities.
Getting started	This module provides an overview of depression; what it is and why it occurs. It introduces the cycle of depression, the basic concept of CBT and TFB cycles.
Getting to grips with mood	This module supports the user to develop a greater understanding of their mood and emotions. To reflect on how their thoughts, physical reactions and behaviour are all interconnected in affecting how they feel.
Spotting Thoughts	The objective of this module is to increase user awareness of unhelpful thinking patterns, to spot distorted thinking and thinking errors and examine outcomes of negative thought cycles.
Boosting behaviour	Boosting behaviour is a practical module aimed at supporting the user to identify behaviour traps. To plan activities that create a sense of pleasure or achievement and identify exercises that will target physical reactions to distress.
Challenge Your Thoughts	This module focuses on identifying hot thoughts and thinking errors, and supports the user to develop a more balanced alternatives to negative thinking patterns.
Core Beliefs	This module supports the user to identify and challenge negative core beliefs that underline negative distorted thinking. It encourages users to find a balanced alternative to unhelpful core beliefs.
Bringing it all Together	This module facilitates users to reflect on the knowledge they have acquired, the skills they have learned and how they are going to progress forward with a focus on staying well and maintaining social support.

2.4.2. Support during treatment

Each participant was assigned a supporter who monitored their progress throughout the trial. All supporters were trained supporters working with Aware who received further training in the Space from Depression and on how to deliver feedback. Where participants discontinued treatment, after one missed sessions the supporter sent a reminder message to the participant by email. If after one further week the participant had not responded to the supporter they were considered to have dropped out.

2.5. Data collection and measures

Quantitative and qualitative measures were employed to investigate users' experience with the internet-delivered depression intervention.

The Beck Depression Inventory (BDI-II) (Beck et al., 1996) is a reliable and validated 21-item questionnaire developed for the assessment of depressive symptoms that correspond to the criteria for depressive disorder diagnosis, as outlined in The American Psychiatric Associations Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition (DSM-IV). Each item is scored on a scale from 0 to 3. The scale designates levels of severity, minimal (0 − 13); mild (14–19); moderate (20–28); and severe (29–63). The BDI-II was administered pre-treatment to provide a baseline measure of depressive symptoms. The BDI-II was administered at baseline and post-treatment.

The Satisfaction with Treatment (SAT) measure (Richards et al., 2013) was administered, at week 8, post-treatment. Descriptive statistics were used to report on the data from the quantitative questions on the SAT measure. The satisfaction measure also contains two questions asking to describe what participants most and least liked about the online treatment. A number of other qualitative questions were administered to provide further insight into the user experience. See Appendix A for a complete description of questions.

2.6. Data analysis

Demographic information and characteristics of respondents was summarised using descriptive statistics. The Beck Depression Inventory (BDI-II) was completed at baseline and scores were calculated to understand the severity of symptoms across the sample.

Data from the first 15 quantitative questions (see Appendix A) was correlated and analysed; the means and standard deviations were summarised to provide an overview of the level of satisfaction with the overall programme, specific modules and specific elements of the modules.

A thematic analysis of qualitative data (13 questions: Appendix A) was conducted by a researcher at SilverCloud Health (TM), to identify common themes and patterns in relation to specific modules and aspects of the treatment programme, and provide further insight into user experiences. Following a comprehensive analysis and interpretation of raw data, initial themes were generated, coded (TM) and these were reviewed by a co-researcher (DR) who has experience in qualitative analysis of this type. The objective of this process was to identify and validate key patterns within a number of themes across a dataset (Braun and Clarke, 2006). Results are discussed in the context of the research questions and in relation to implications for service providers, clinicians and service users alike.

3. Results

Results from the data analysis were organised into five domains including: the user-experience, platform functionality, online support, content modules and intervention impact (see Table 3). It is interesting to note at this point that participants who received treatment demonstrated statistically significant improvement on measures of depression from pre- to post-treatment and these gains were maintained at 6-month follow up, for more details see the published report (Richards et al., 2015).

Table 3.

Domains.

Domains	Description
The user experience	This section reports on initial attraction and user satisfaction with the programme. It identifies whether users found it informative and helpful. Investigated what users liked most and least, and their overall experience. Users also compared CCBT to previous treatments.
Platform functionality	This section reports on how easy the users found the programme to use and how happy users were to access their treatment online.
Online support	This section reports on how supported the users felt, their perceptions of having an online supporter and experience of sharing information with them.
Content modules	This section reports individual module ratings, the content and aspects liked best and least and improvement suggestions made by users. An outline of the individual modules' content is provided in the method's section.
Intervention impact	This section reports the potential short-term and long-term effects of the programme as discussed by users.

3.1. The user experience

This first domain reports respondents experience of and satisfaction with the online treatment.

3.1.1. Initial attraction to the programme

Respondents (n = 77, 27%) reported having been initially attracted to the programme in their own words due to its accessibility and flexibility (see Table 4: Appendix B), that it had been recommended by a trusted source such as their GP/family/friend/or the Aware charity.

“The fact that I could do it at my own pace and that I wasn't under any added pressure... not having something else that I had to do. I like being able to log on at any time of day or night when I can find time to sit and give it my attention”

Other reasons given by respondents included self-management (learning new skills and personal development), that they stumbled across it when searching for help, an alternative method having tried other treatments, anonymity, CBT and the idea of having a supporter.

“found it online and thought it would help me with my personal problems”

3.1.2. Satisfaction with treatment programme

In response to the quantitative question on satisfaction, the majority of respondents (M = 3.96; S.D. = 0.96) were satisfied with the treatment programme (see Table 5: Appendix B). Participant's found the programme informative (M = 4.25; S.D. = 0.83), helpful with any difficulties they were experiencing (M = 3.93; S.D. = 0.95) and found the treatment programme helpful in general (M = 3.15; S.D. = 0.82).

When asked what they ‘most liked’ about the online treatment programme the majority of respondents reported its accessibility and flexibility (n = 79, 28%). Other key areas liked included having a supporter, the engaging and user friendly content, and the CBT techniques, anonymity, interactive tools and activities, the fact that it was self-help and the personal stories (see Table 6: Appendix B).

“That I was able to access it at a time convenient to me and from the comfort of my own home,” “I really liked the feedback as it was motivating. And I have never spoken to anyone about how I feel,” “It was user friendly, easy to do and understand,” “Teaching me that things I think can be connected to how I feel.”

When asked what they ‘least liked’ about the online treatment programme (n = 41, 15%) some participants reported that the programme did not meet their individual needs (see Table 7: Appendix B).

“Reading about issues is not enough for me, I have to talk but I can't bring myself to do it,” “Finding my way around the program. Knowing where I was when I opened the program”

There were a number of key themes that arose when respondents were asked to use three words to describe their experience of the programme (see Table 8: Appendix B). These included that it was effective/life changing, positive/enjoyable, beneficial/valuable, informative, encouraging/motivating, insightful/awareness and supportive.

3.1.3. Comparison with previous treatments

Of those who had received previous treatments 44% reported the online treatment to be better or much better (n = 74; M = 3.43, S.D. = 1.20), while 34% felt it was about the same (n = 57) in comparison to previous treatments received (see Table 9: Appendix B).

3.2. Platform functionality

The majority of respondents agreed that they found the programme easy-to-use (M = 3.80; S.D. = 0.99). The majority agreed/strongly agreed that they were happy to use the computer to access treatment (M = 4.15; S.D. = 0.88). Results indicate that overall users had a positive experience in terms of the programme functionality and that they were happy to access their treatment on the computer (see Table 10: Appendix B).

3.3. Online support

The majority of respondents agreed feeling well supported as they worked through the programme (M = 4.03; S.D. = 0.87) (see Table 11: Appendix B). Participants were further asked in an open ended way about their experience of being supported by a trained volunteer as they progressed through the modules. Table 12 (Appendix B) shows the results of thematic analysis of those responses. Respondents found this experience to be supportive and helpful (n = 86, 31%), encouraging and motivating (n = 41, 15%) and that their supporters provided helpful guidance and feedback (n = 34, 12%).

“I found this part of the course was the best aspect for me as the volunteer was extremely understanding and helpful even when I had lost interest she continued to try and help,”. “Reading the feedback is certainly an incentive to check in with the programme and the positive feedback is encouraging”

“Without human support… there would be no real push on to finish and complete the course “Not as confident about its effectiveness”

Most respondents reported sharing information with their supporter, 50% (n = 141) and the reasons identified for doing so included: getting the greatest benefit from the programme (21%, n = 30), to receive feedback/guidance (21%, n = 30) and feeling it was easy to open up within this environment (13%, n = 19).

“To achieve optimum benefit from the programme it seems to me that it must be a two way process” “It was great to be able to have someone see what you were thinking from a different point of view and provide feedback”

A large proportion of users (n = 115, 41%) reported not sharing much with their supporter and attributed this to not being able to engage with the programme (n = 18, 16%), privacy (n = 16, 14%), lack of time (n = 15, 13%) and uncertainty regarding what or how much to share (n = 15, 13%).

“I'm a fairly private person I find it a bit hard to open up” – Privacy.

3.4. Content modules

Users were asked to rate each module on a 10-point scale, 0 being not good, 5 being neutral and 10 being great. The highest rated module was ‘Tune in 2: Spotting Thoughts’ (M = 7.45; S.D. = 2.06), which is a module on identifying negative automatic thoughts, and the lowest rated module was the final module ‘Bringing it all Together’ (M = 6.37; S.D. = 1.94), which focuses on planning for relapse prevention. It is noted however, that respondents rated each of the modules above average (M = 7.02; S.D. = 2.07). These results indicate that on average respondents experience of each module was positive.

In response to an open-ended question, the modules liked best included ‘Spotting Thoughts’ (recognising and identifying negative automatic thoughts) (n = 45, 16%), ‘Core Beliefs’ (identifying underlying central themes of influence) (n = 35, 12%) and ‘Challenge Your Thoughts’ (finding evidence for and against and reframing negative thoughts) (n = 32, 11%).

“Spotting thoughts for me as I just found this a great way of looking at what I was thinking first instead of dwelling on them as I had a habit of doing” “Core Belief section as this has helped me challenge some of my beliefs and look at things in a more balanced way”

Given the sample size the numbers of individuals responding to modules they least liked are relatively insignificant. In fact, 42% (n = 74) of the total respondents to this question (n = 175) reported that there was no module they disliked and a further 14% (n = 25) were not sure.

3.4.1. Modules completion

When asked whether they completed all of the modules 27% of respondents reported that they had, while the majority (n = 136, 48%) reported not completing all modules. The remainder did not answer this question. The predominant reason given for not completing modules was time restrictions (n = 69, 51%); respondents either did not have time due to personal circumstances or needed longer to complete the programme. Other respondents reported a loss of interest or motivation (n = 18, 13%), or personal difficulty (n = 17, 13%).

“No I didn't finish it. I was quite busy and didn't give enough time to this during the supported period”. “No just lost interest in doing anything once I started the programme and didn't log on as much as I should have and I regret it now”.

In response to an open-ended question, users' suggestions on how to improve the programme referred to its structure (n = 35, 12%) reporting having experienced difficulties with navigation. Eleven per cent of respondents (n = 30) were not sure what improvements they could suggest, 6% (n = 17) of respondents would have liked more contact with their supporter, 5% (n = 15) suggested more personalised feedback/guidance and more time (5%, n = 14).

Key components of the programme that respondents reported to like best are displayed in Table 13 (Appendix B). These included the core CBT activities, mood monitoring, setting goals, the videos, psycho-educational content, take home points, and mindfulness.

3.5. Intervention impact

Respondents were asked a number of questions relating to the impact they felt the programme would have on their lives (see Table 14: Appendix B). Over half of the sample agreed that they felt the treatment they received would have a long lasting effect (M = 3.62; S.D. = 1.04, n = 149). Respondents were asked to rate, on a 10-point scale, whether they had noticed any changes in any area of their lives as a result of the programme. On average respondents reported a positive change to at least one area of their lives (M = 6.79; S.D. = 1.80).

When asked in an open-ended question whether there was anything in particular they noticed in relation to changes resulting from taking part in the programme, respondents reported on having developed coping strategies (n = 78, 28%), CBT specific techniques such as spotting and challenging thoughts (n = 42, 15%), lifestyle change (n = 36, 13%), and improved mood (n = 35, 13%). Other developments reported included improvement in mood, positive attitude, acquisition of knowledge, improved self-esteem and self-awareness (see Table 15: Appendix B).

“My coping strategies definitely improved. Where previously I might have become panicky or overwhelmed with a stressful situation, I'm getting better at taking a deep breath and reacting more calmly”. “I've also become better at spotting negative irrational thinking where I used to blame myself for situations outside my control, or feel that because I was having problems with one aspect of my life that my entire life was failing”

Respondents were also asked to predict how they thought the treatment programme would impact on their future ambitions or aspirations (see Table 16: Appendix B). Respondents reported having a more hopeful and positive outlook for the future (n = 52, 19%). Other impacts included providing a good foundation to build on and revisit, development of practical tools and coping skills, boosting confidence, self-esteem and awareness, help in everyday life, in changing unhelpful thought patterns and in setting goals and planning for the future.

4. Discussion

The results from the satisfaction and user-experience questionnaires are promising, indicating that in general service-users are satisfied with accessing an iCBT intervention. Some participants felt the treatment did not meet their individual needs as they found it difficult to get motivated and engage with the programme. These findings are discussed, as are suggested improvements to drive user motivation and engagement.

4.1. Satisfaction & acceptability

Respondents who had previously received another form of treatment found the iCBT treatment to be about the same or better when compared to face-to-face or a combination of both face-to-face therapy and medication. Furthermore, those that had previously received only medication as a treatment found the internet-delivered treatment to be much better. This may reflect a preference for psychological therapies as discussed in previous literature (Van Schaik et al., 2004). The current results may reflect the needs and preferences of individuals who have not accessed face-to-face support due to physical or psychological barriers. This would suggest that an internet-delivered treatment has the ability to reach and be an acceptable form of treatment for individuals who may be on waiting lists, unwilling or unable to access traditional counselling services due to physical or geographical constraints from services.

Users highlighted effective and personalised tools and content and a non-judgemental supporter as key aspects. These results are in line with previous research that promotes the provision of an online supporter in driving user engagement and tailored content to meet specific user needs and improve satisfaction (Richards and Timulak, 2013).

4.2. Perceived efficacy

The literature suggests that user satisfaction is correlated with clinically significant symptom changes (Ankuta and Abeles, 1993). Users described their experience to be effective, positive and beneficial suggesting an overall sense of efficacy. Other words used to describe the user experience included informative, encouraging and motivating, which supports the idea of a low-intensity self-management tool being empowering (Wright, 2002). The majority of the sample felt that the programme had led to changes in at least one area of their lives and that these would have a lasting effect. The main changes participants noticed were the enhancement of coping strategies and the application of specific CBT techniques; indicating effective translation of evidence-based theory in an online environment. They reported noticing lifestyle changes around routine, activities, diet and exercise. These changes are related to improved functioning which in previous literature has been correlated with overall satisfaction, as functional impairment has been highlighted as one of the most distressing symptoms of depression for patients (Ankuta and Abeles, 1993, Hollon et al., 2002). Improved mood, self-esteem and attitude were also reported. These results indicate that the programme was perceived as effective across both social and emotional domains of participants' lives.

Participants felt that they had a more positive and hopeful outlook for the future, that they had a good foundation to build upon whether it was to revisit the information or acting as a stepping stone to accessing further support. This would further support research on the implementation of iCBT as a low-intensity treatment; to support individuals presenting within the mild to moderate range or those who are not ready, unable or unwilling to access traditional services.

4.3. Flexibility & accessibility

Flexibility and accessibility were most liked which is consistent with previous literature (Ritterband et al., 2003). Users are happy to access a treatment that overcomes common barriers; reduces appointment related pressure and perceived stigma, accommodates personal needs and increases autonomy (Kessler et al., 2001). Our results demonstrate the importance of providing innovative services to increase reach and access for service users and providers.

Socio-demographic characteristics are largely representative of previous studies in this field in terms of age and gender. One quarter of the current sample were male, typical of the predominance of females presenting with symptoms of depression (Meyer et al., 2009). Older age adults above the age of 63 and individuals of low-socio economic status are underrepresented in this study. This may be due to access to computers and internet or computer literacy within these populations. Eighty-three percent of users reported being very confident with using computers and the internet, this again may be representative of those attracted to an internet-delivered treatment or possibly an increase in computer literacy among the general population. The majority of participants had a university degree and were in fulltime or part-time employment/students, indicating that they were active members of the community. It is possible that motivation to self-manage is not a difficulty within a well-functioning population, or that use of technology is common within work and educational environments. Further research into these underrepresented populations may inform as to how their needs may be met in an online environment. This may be relevant to help seeking behaviours among this population, highlighting the possible need for promotion through more traditional methods to aid the user journey and improve reach and access.

4.4. Design & development

Participants found the intervention easy to use and were happy to access their treatment online. Interactive activities were identified as one of the best aspects of the programme. The purpose of these activities is to facilitate users to put content and knowledge into action, which suggests it is an effective way to support and integrate learning. The mood monitor was one of the best tools identified, providing a visual representation that facilitates users to identify patterns and relationships. This activity of logging and monitoring, over time, supports the user to identify key areas for change.

One of the most liked aspects of the programme was the engaging and user-friendly content that may represent a person-centred experience from the user perspective and specific CBT techniques. In contrast, some respondents reported the programme to be difficult to navigate and to remember where they had last been while previously logged in. Impersonal content was also highlighted as one of the least liked aspects of the treatment; this indicates that some users perceived the treatment not to be person-centred enough. This could be a reflection of individual expectations of the type of support a low-intensity self-management tool can provide. Secondly it may indicate the need for further development of multiple options or pathways to suit responses or specific user experiences. While understanding the components users least liked about the intervention is important for future development, it must be noted that the number of individuals who least liked these elements were relatively low.

The best liked modules incorporated core elements of CBT theory and practice. The main reasons participants identified for liking these modules included finding them beneficial, helpful in rationalising thoughts and overcoming negative thought patterns. Participants felt as though these modules had created awareness and provided new insight into their depression related difficulties. They felt the content of these modules was relevant to their specific needs and really reached the core of their problems. This seems to suggest that the current content met the needs of most users, was relevant to them and promoted change.

Participants made a number of suggestions to improve their experience with the programme. The main suggestions referred to the programme structure, specifically to the need to improve navigation through the programme and the layout. This highlights the importance of ensuring content delivered online is user-friendly; this may be particularly important for more challenging content, which may benefit from the incorporation of interactive tools and activities to cater for different learning styles and promote usability and engagement.

4.5. Time

Almost half of the sample reported that they had not completed all of the modules. The main reason given was that they did not complete them due to time constraints. This included a lack of time due to other commitments or personal circumstances. Others felt as though they did not have enough time to complete the programme and reported having to rush through content between reviews and not having the time to engage in activities and reflect on the content in detail. These results tie in with user suggestions regarding more time between reviews, as people wanted to make the most of their supporter while they were being supported but felt rushed and under pressure to cover everything. A number of respondents suggested more contact with their supporter; either more regular feedback or a follow-up post treatment to encourage engagement with the programme. Needing more time with the treatment while being supported demonstrates the demands of the treatment and perhaps the need for further flexibility to meet individual needs (Richards and Timulak, 2013). Future interventions may benefit from flexibility around review deadlines, providing options to users in order to accommodate individual needs and preferences. Further research may inform the appropriate level, duration and flexibility of support required by users.

4.6. Supporter

Characteristics of the programme that were identified as most liked included having a supporter to provide motivation, guidance and feedback. Users felt well supported, stating that their contact with a supporter was supportive and helpful, encouraging and motivating with effective guidance and feedback. A number of users also found their supporter to be impersonal. This may reflect the quality of reviews perceived by service-users and suggests the possible need for supporters to provide individually tailored and person-centred feedback. It may also highlight the need for further training with supporters on how to make their feedback personal, in being familiar with the programme content and being able to relate to and contextualise their feedback and guidance.

Without a supporter users anticipated that they would be less likely to engage and complete the programme, and felt that it would be a less beneficial experience overall. A number of users felt the programme would be the same without a supporter. This highlights individual preference which has been accommodated for with users having the option to share as little or as much as they would like. A large sample of respondents did not share much with their supporter; reasons for not sharing included not engaging with the programme, privacy, time and uncertainty around what or how much to share. Participants who did share stated that they found it easy to open up to their online supporter, which may reflect an enhanced sense of anonymity unique to online treatment (Efstathiou, 2009).

Suggested improvement around how support is delivered included more personalised feedback and guidance. This is important to supporter training as the supporter has been described as motivating and plays a key role in users engaging with and completing the programme. There may be a need to incorporate information and definition around self-directed learning, guided support and the supporter functionality into the welcome message. The quality and interpretation of feedback and information may have implications on making informed choices while working through the programme independently (Richards and Timulak, 2013). There is a question here as to whether satisfaction with supporter feedback is related to the background of the supporter and their level of programme specific training, or whether it is related to user preferences and expectations.

4.7. Limitations

Participants who had not completed all of the modules were included in the final analysis; however, it is not clear as to whether they intended and in fact went on to complete all of the modules post-treatment, or whether these individuals had lost motivation and disengaged. The experience of individuals who did not engage with the intervention may not be fully represented. This raises the question as to whether these individuals would have affected the overall satisfaction measures within the study. Comparisons with previous research, which had small sample sizes, may indicate the need for further research to make accurate inferences. Self-administered questionnaires inevitably entail a limitation regarding interpretation of questions and results.

5. Conclusion

This study has demonstrated the potential for internet-delivered interventions to provide satisfactory, acceptable and effective low-intensity treatments to individuals living with depression. The accessibility and flexibility unique to an online environment may increase the ability to overcome physical and psychological barriers associated with traditional service delivery, while increasing the overall capacity of mental health services. Future research may inform the user pathway in order to reach underserved clinical populations. There is a need for further comparison with iCBT interventions in order to identify core elements in the development of an effective online intervention. Conclusions from the current research indicate the need for improved format and structure, and increased flexibility related to the level and duration of support provided. Understanding the user experience is central to the successful development and implementation of an evidence-based internet-delivered intervention. The results from this study are encouraging for the implementation of the internet-delivered cognitive behaviour programme, Space from Depression, for treatment of symptoms of depression within a community population.

Appendix A. Satisfaction with treatment measure (SAT)

15 questions in the SAT measure:

Quantitative Qs - comparing to previous treatment, overall satisfaction, whether they found it informative, helpful and how helpful, whether users found it easy-to-use, happy to access treatment through a computer, whether they felt well supported and whether they perceived the treatment would have a lasting effect. Users were also asked to rate each module.

Scales:

*Better/not good – 5 point Likert scale

• •Much/a little better = 5/4
• •About the same = 3
• •Not quite/not at all good = 2/1

*Satisfied – 5 point Likert scale (Satisfied/Dissatisfied)

• •Satisfied/Very Satisfied = 4/5
• •Neutral = 3
• •Dissatisfied/Very Dissatisfied = 2/1

*Agree/Disagree - 5 point Likert scale

• •Agree/Strongly Agree = 4/5;
• •Neither agree nor disagree = 3;
• •Disagree/Strongly Disagree = 2/1;

*Helpful/Unhelpful - 4 point Likert scale

• •Quite/Very helpful = 3/4;
• •Not really helpful/Not at all helpful = 2/1

*Ratings

• •10-point scale, 0 being not good, 5 being neutral and 10 being great.

13 Qualitative Questions

SAT

• 1.SAT - Any Previous Treatment?
• 2.SAT - What treatment did you receive?
• 3.SAT - How did this online treatment compare to previous treatments?

*Better/not good – 5 point Likert scale

• •Much/a little better = 5/4
• •About the same = 3
• •Not quite/not at all good = 2/1

• 4.SAT - How would you rate your overall satisfaction with the programme?

*Satisfied – 5 point Likert scale (Satisfied/Dissatisfied)

• •Satisfied/Very Satisfied = 4/5
• •Neutral = 3
• •Dissatisfied/Very Dissatisfied = 2/1

• 5.SAT - I found this programme informative

*5 point Likert scale (Agree/Disagree)

• •Agree/Strongly Agree = 4/5;
• •Neither agree nor disagree = 3;
• •Disagree/Strongly Disagree = 2/1;

• 6.SAT - I found this programme helpful with any difficulties I am having

*5 point Likert scale (Agree/Disagree)

• •Agree/Strongly Agree = 4/5;
• •Neither agree nor disagree = 3;
• •Disagree/Strongly Disagree = 2/1;

• 7.SAT - Please rate how helpful you found the online treatment programme

*4 point Likert scale (Helpful/Unhelpful)

• •Quite/Very helpful = 3/4;
• •Not really helpful/Not at all helpful = 2/1

• 8.SAT - What did you most like about the online treatment?

*open ended question

• 9.SAT - What did you least like about the online treatment?

  *open ended question

• 10.SAT - I was happy to use the computer to access treatment

*5 point Likert scale (Agree/Disagree)

• •Agree/Strongly Agree = 4/5;
• •Neither agree nor disagree = 3;
• •Disagree/Strongly Disagree = 2/1;

• 11.SAT - I found the online treatment easy to use

*5 point Likert scale (Agree/Disagree)

• •Agree/Strongly Agree = 4/5;
• •Neither agree nor disagree = 3;
• •Disagree/Strongly Disagree = 2/1;

• 12.SAT - I felt well supported as I worked through the programme

*5 point Likert scale (Agree/Disagree)

• •Agree/Strongly Agree = 4/5;
• •Neither agree nor disagree = 3;
• •Disagree/Strongly Disagree = 2/1;

• 13.SAT - Module Ratings – 8 modules

*Rating Scale: 0–10; 5 = Neutral

• 14.SAT - I feel the treatment received will have a long lasting effect

*5 point Likert scale (Agree/Disagree)

• •Agree/Strongly Agree = 4/5;
• •Neither agree nor disagree = 3;
• •Disagree/Strongly Disagree = 2/1;

• 15.SAT - Did you notice any changes in any area of your life as a result of the programme? Indicate negative changes to the left and positive to the right.

*Rating Scale: 0–10; 5 = Neutral.

Qualitative Questionnaire

• 1.
  Have you participated in any online CBT programmes before?

  • a.
    If yes, how does Silver Cloud compare
• 2.What attracted you to the programme?
• 3.Which aspects of the programme did you like best? (E.g.: activities, videos, charts and mood monitor, lists, goal for the week, take home point.)
• 4.Can you use three words to describe your experience of the programme?
• 5.The support of an Aware volunteer is one of the unique aspects of Space from Depression. What was it like for you having this contact with another person, and their support?
• 6.If you did not have their support how would you feel about the programme?
• 7.
  Did you share much?

  • a.
    Why was this?
• 8.Of the modules you worked on, which module did you like best and why?
• 9.Which module(s) did you like least and why?
• 10.Did you complete all the modules? If not, why not?
• 11.Do you have suggestions to help us improve the programme or the site? Any other comments?
• 12.Was there anything in particular you noticed? (e.g.: mood, coping strategies, attitude, knowledge, daily routine or activities, lifestyle, self-esteem, body image)
• 13.How do you think this programme will impact on your future ambitions/aspirations?

Appendix B.

Table 4.

Reasons for participants initial attraction to the treatment programme.

Attraction to the programme	N = 281	% sample
Accessibility & flexibility	77	27.40
Recommended by a trusted source/multi-media	57	20.28
Stumbled across it while searching for help	33	11.74
Self-management: acquisition of new skills/knowledge/personal development	33	11.74
Alternative method	17	6.05
Anonymity	15	5.34
Cognitive Behaviour Therapy (CBT)	9	3.20
Supporter	4	1.42
Content	2	0.71

Table 5.

Satisfaction with the programme.

Overall satisfaction with the programme	N = 281	% sample
Mean	3.96
SD ±	0.96	SD ± (3.00–4.92)
Very satisfied	79	28.11
Satisfied	112	39.86
Neutral	44	15.66
Dissatisfied	12	4.27
Very dissatisfied	8	2.85

Note *5 point Likert scale: Satisfied/very satisfied = 4/5; neutral = 3; dissatisfied/very dissatisfied = 2/1.

Table 6.

What respondents most liked about the overall treatment.

Most like about the online treatment	N = 281	% sample
Accessibility & flexibility	79	28.11
Supporter	48	17.08
Engaging & user friendly content	35	12.46
CBT techniques	33	11.74
Anonymity	25	8.90
Interactive tool & activities	23	8.19
Self-help	21	7.47
Personal stories	19	6.76

Table 7.

What respondents least liked about the online treatment.

Least like about the online treatment	N = 281	% sample
Did not match the needs of the user	41	14.59
Format/content delivery	35	12.46
Impersonal	26	9.25
Needing more time (programme/support)	23	8.19
Complicated content	11	3.91
Pressure to answer feedback/ques.	9	3.20
Technical difficulties	8	2.85
A lot of work	6	2.14
Mindfulness	3	1.07

Table 8.

Respondents use of 3 words to describe their experience.

Three words to describe experience of the programme	N = 281	% sample
Effective/life-changing	95	33.81
Positive/enjoyable	76	27.05
Beneficial/valuable	57	20.28
Informative	57	20.28
Encouraging/motivating	52	18.51
Supportive	44	15.66
Insightful/awareness	43	15.30
Thought provoking/challenging	21	7.47
Unhelpful	15	5.34
User friendly	13	4.63
Empowering	12	4.27
Frustrating/disappointing	11	3.91
Difficult	10	3.56
Different	4	1.42

Table 9.

Comparison with previous treatments.

Yes - How did this online treatment compare to previous treatments?	N = 168	% sample
Mean	3.43
SD ±	1.20	SD ± (2.23–4.63)
Much better	42	25.00
A little better	32	19.05
About the same	57	33.93
Not quite as good	21	12.50
Not at all good	12	7.14

Note *5 point Likert scale; much/a little better = 5/4; about the same = 3; not quite/not at all good = 2/1.

Table 10.

Respondents experience of platform functionality.

	N = 281	% sample
I found this programme easy-to-use
Mean	3.80
SD ±	0.99	SD ± (2.81–4.79)
Strongly agree	63	22.42
Agree	111	39.50
Neutral	54	19.22
Disagree	20	7.12
Strongly disagree	7	2.49
I was happy to use the computer to access treatment
Mean	4.15
SD ±	0.88	SD ± (3.27–5.03)
Strongly agree	102	36.30
Agree	104	37.01
Neutral	37	13.17
Disagree	7	2.49
Strongly disagree	4	1.42

Table 11.

Respondents experience of support through the programme.

I felt well supported as I worked through the programme	N = 281	% sample
Mean	4.03
SD ±	0.87	SD ± (3.16–4.9)
Strongly agree	75	26.69
Agree	128	45.55
Neutral	39	13.88
Disagree	8	2.85
Strongly disagree	5	1.78

Table 12.

Respondents experience of having a supporter.

The support of an Aware volunteer is one of the unique aspects of the programme. What was it like for you having this contact with another person, and their support?	N = 281	% sample
Supportive & helpful	86	30.60
Encouraging & motivating	41	14.59
Helpful guidance/feedback	34	12.10
Didn't make use	32	11.39
Good to know someone is there	28	9.96
Impersonal	21	7.47
Easy-to open up	14	4.98
Not helpful	12	4.27
Personal	8	2.85
Felt under pressure	4	1.42
Not as good as face-to-face	2	0.71

Table 13.

Respondents reports of the programme aspects liked best.

Aspects of the programme liked best	N = 281	% sample
Activities	72	25.62
Mood monitor	59	21.00
Goal for the week	43	15.30
Videos	39	13.88
Psycho-educational content	33	11.74
Take home points	29	10.32
Mindfulness	29	10.32
Charts	23	8.19
Lists	21	7.47
Personal stories	19	6.76
Journal	13	4.63
TFB cycles	11	3.91
Module summary (print)	10	3.56
Everything	10	3.56
Supporter	6	2.135
Structure	3	1.07

Table 14.

Respondents reports of whether the treatment will have a lasting effect.

I feel the treatment received will have a long lasting effect	N = 281	% sample
Mean	3.62
SD ±	1.04	SD ± (2.58–4.66)
Strongly agree	52	18.51
Agree	97	34.52
Neutral	75	26.69
Disagree	17	6.05
Strongly disagree	13	4.63

Table 15.

Respondents reports on changes they have noticed.

Was there anything in particular you noticed? (e.g.: mood, coping strategies, attitude, knowledge, daily routine or activities, lifestyle, self-esteem, body image)	N = 281	% sample
Coping strategies	78	27.76
CBT techniques	42	14.95
Lifestyle change	36	12.81
Mood improved	35	12.46
Positive attitude	30	10.68
Knowledge	21	7.47
Self-esteem	21	7.47
Awareness	16	5.69
Motivation	10	3.56
Positive body-image	6	2.14
Everything	5	1.78
Unhelpful	4	1.42
Behaviour	1	0.36
Relationships	1	0.36

Table 16.

Respondents reports on the impact of the programme.

How do you think this programme will impact on your future ambitions/aspirations?	N = 281	% sample
Hopeful/positive outlook	52	18.51
Good foundation to build on/revisit	39	13.88
Practical tools/coping skills	21	7.47
Boost confidence/self-esteem/awareness	21	7.47
Not sure	19	6.76
Help in everyday life	17	6.05
Change unhelpful thought, feeling and behaviour patterns	17	6.05
No impact	16	5.69
Setting goals/planning	10	3.56