Abstract
Background
Depression and anxiety is a major public health problem, in Sweden and internationally. Internet-based interventions are increasingly acknowledged as promising approaches for individuals with varying degrees of mental health problems. We present findings from the implementation of myCompass, a fully automated self-guided intervention of Australian origin, in a Swedish context.
Methods
We (i) share our experience of the E-health study platform (i.e., regarding security aspects, functionality) to which the myCompass intervention was linked, and (ii) report findings from the empirical evaluation of myCompass (i.e., prerequisites, execution, study outcomes), in a community sample of individuals (N = 837) reporting mild-to-moderate levels of depression, anxiety and stress. Outcomes were calculated with repeated measures ANOVA and linear mixed models.
Results
The E-health study platform proved to be an efficient tool enabling randomization, informed consent and evaluation to be administered in a fully automated manner. The study rendered substantial interest initially with 1207 individuals enrolling, however it failed to maintain engagement of those enrolled with only few participants logging in more than once or twice following registration. A smaller subgroup of “active users” (n = 35) had a markedly higher activity in the program, however their treatment results were not significantly better than those of the control group.
Conclusion
Based on the large number of dropouts and also modest use of the intervention overall, only tentative speculations can be made regarding its effectiveness in a Swedish context. The number of individuals remaining active in the intervention is much more limited that the number of individuals initially signing up. Moreover, the transportation of interventions across countries and cultures may need more careful consideration, and pilot-trials before attempting large-scale trials are recommended.
Trial registration
MyCompass was retrospectively registered at ClinicalTrials.gov. NCT03659630 September 3rd 2018, and was given the protocol ID 2015/1268–31/2 + 2016/88.
Keywords: Internet-based intervention, myCompass, Depression, Anxiety, Stress
Background
Mental health problems are one of the leading causes of disability internationally [1]. Globally, an estimated 17% of the adult population has been affected by at least one mental disorder during a 12 month period [2]. Specifically, depression and anxiety constitute a major public health issue, with depressive disorders alone being one of the leading causes of disability worldwide [3]. Depression and anxiety are commonly associated with a range of negative consequences in the daily life for the afflicted individual (e.g., reduced life quality, social and work related functional impairments, risk of somatic and psychiatric comorbidity). They are also associated with high societal costs in the form of health care consumption and loss of production [4]. Moreover, psychiatric diagnoses including depression and anxiety is the most common reasons for long-term sickness absence in developed countries [5].
Internet-based interventions
While there are evidence-based psychosocial interventions for several common mental disorders (e.g. anxiety and depression) it is unlikely that traditional face-to-face interventions alone will solve the disease burden of mental disorders. Despite the availability of different types of traditional treatments for various mental health conditions, a common finding is that few people seek care [6]. This might be due to deficient knowledge regarding symptoms, lack of access to health care, fear of stigma, cost of treatment, and a will to handle problems oneself [7–10]. Also, the sheer number of individuals who suffer from mental illnesses calls for complementary approaches and prevention strategies. Internet based interventions could help overcome several of these obstacles since they are accessible, flexible, anonymous, cost-effective and not confined to a particular clinic.
Internet based interventions are heterogeneous, with respect to: (i) type of problem being targeted, (ii) how active the user is expected to be, (iii) technical properties of the platform used, and (iv) ways in which the intervention is delivered and managed [11]. Some interventions are technologically simple, consisting of texts with accompanying exercises, while others include more advanced solutions such as interactive exercises, smartphone applications, animated videos and the possibility to chat with a therapist [11–13]. Furthermore, some interventions are designed to provide tailored but basic information on one’s habits, such as the drinking feedback website www.checkyourdrinking.net [14], while others provide a more full-scale treatment including hefty text materials and exercises accompanied with extensive support via e-mail, chat and scheduled telephone calls [11].
Most internet based interventions to date are developed to target a specific condition (e.g., depression or social phobia), but there are also transdiagnostic interventions designed to focus on problems common across disorders [15]. Barak, Klein [11] proposed three broad categories of internet-based interventions: (i) interventions mainly aimed at education, (ii) self-guided interventions without a therapist, and (iii) therapist-guided interventions. Another categorization is based on whether the intervention could be regarded as general support or a regular health care intervention (i.e., which in most cases requires that the caregiver keeps a journal or chart documenting the status of the patient and all actions taken on behalf of the professional). The line between “advice and support” versus “health care” is not always clear-cut, and legislation may differ between countries. In Sweden, therapist-guided digital interventions are usually regarded as regular health care.
Since the implementation of the earliest internet-based intervention in the late 1990’s, the number of publications on their effectiveness with regards to mental health conditions has increased steadily [16]. Multiple studies have demonstrated that therapist-guided interventions could yield treatment results comparable to face-to face treatments for a wide variety of conditions across trials [17–21]. For example, a systematic review and meta-analysis of 13 studies on both somatic and psychiatric conditions (e.g., social anxiety disorder, panic disorder, tinnitus) demonstrated that guided internet-delivered cognitive behavior therapy (ICBT) and face-to-face CBT demonstrated similar effects (Andersson et al., 2014). A recent meta-analysis of individual change in 24 RCTs with internet-based interventions stated that the effects are not just statistically significant, but also clinically meaningful [22]. But internet-based interventions, as well as face-to-face treatments, have also been noted to create negative side effects for a small minority of participants [23]. One possible direction forward proposed by Muños et al. (2018) is to offer “digital apotecharies” where a plethora of evidence-based interventions for a wide range of problems is offered to the patient [24].
As one of several options, self-guided interventions could play an important role in preventing and treating psychological ill-health. Self-guided interventions have generally provided significant results, however not on par with face-to-face treatments [20, 25, 26]. Self-guided interventions are associated with specific advantages including the possibility to provide evidence-based interventions to an indefinite number of clients to a low cost. It could also suit participants who are reluctant to disclose details about their private life to therapists, as well as participants who value the flexibility self-guided interventions can provide. One example is MoodGym, targeting depression, with an estimated 1.2 million users in five countries [27]. Some patients might prefer self-guided over therapist guided interventions [28], and self-guided interventions could complement more traditional alternatives by providing self-help tools well suited for individuals with mild to moderate problems. Internet-based interventions, particularly self-guided ones, commonly suffer from high rates of attrition [29] however, even though there are exceptions [30].
myCompass
MyCompass is a self-guided intervention of Australian origin, specifically designed for individuals with mild to moderate symptoms of stress, depression and anxiety (see Proudfoot, Clarke [31] and Harrison, Proudfoot [32] for detailed descriptions). This patient group could be particularly important to target, given that they represent approximately three quarters of individuals with mental health problems according to the Organisation for Economic Co-operation and Development [33]. Also, most internet-based interventions have focused on this degree of impairment (Andersson & Titov, 2014). The myCompass intervention is based on cognitive behavioral therapy, and encompasses aspects of problem solving therapy, interpersonal psychotherapy, and positive psychology (Proudfoot et al., 2013). In the sole empirical evaluation of the program published to date, community members (N = 720) were randomized into one of three fully automatized interventions: (i) myCompass (ii) a control intervention, and (iii) a wait list. Assessment of participants’ symptoms and functioning was done at baseline, post-intervention and at three months follow-up. Compared to the control groups, participants in the myCompass group demonstrated significant improvements with regards to symptoms of depression, anxiety, stress, and also improved work capacity and social function [31]. At the end of the intervention, participants in the myCompass group had mean symptomatic levels approaching levels in the general population, showing stability 90 days later [31]. This indicates that the program can contribute to significant health improvements, with decreased levels of mental ill health. A follow-up analysis concluded that myCompass was highly cost-effective and could help reduce demands on regular psychiatric services [34]. There was a high level of dropout in the myCompass group, however, and varying rates of engagement among the participants [31]. Given these preliminary positive findings, evaluations of the program in additional contexts are needed.
Methods
Aim
The aim of the current article is twofold: (i) to describe characteristic features of the E-health study platform (i.e., regarding security aspects, functionality) to which the myCompass intervention was linked, and (ii) to report findings from the empirical evaluation of myCompass (i.e., prerequisites, execution, study outcomes), in a community sample of individuals in Sweden reporting mild-to-moderate levels of depression, anxiety and stress. Our goal with describing both the platform and the intervention is to contribute with knowledge that may be valuable to researchers and clinicians designing and implementing internet-based interventions targeting various patient groups across different cultural contexts.
Background
Based on the overall successful results in Australia, an initiative was taken by the Swedish Association of Local Authorities and Regions and the Swedish Ministry of Health and Welfare to evaluate novel treatment forms and targeted mental health interventions. MyCompass was translated into Swedish and launched in December 2015 with the aim to provide an easily available support intervention independent of the health care system for individuals with mild to moderate symptoms of stress, depression and anxiety. A research study to explore its effectiveness was designed. The Regional Ethical Review Board in Stockholm approved the study (# 2015/1268–31/2 + 2016/88).
The E-health study platform
The myCompass intervention was connected to the digital E-health study platform (ehalsostudie.se). This platform allows for a completely automated study process from enrollment to evaluation, with no assistance from the research team.
Security aspects
The platform functions as a type of “gateway” for different types of interventions. Each study is hosted on a separate database, and no web application can reach data that does not belong to a particular study. Technically, the platform was developed using two layers: a web-based application and a database, which in turn are hosted on two separate servers. These two layers are separated by a firewall that overlooks the traffic in between layers. Another unique feature is that the E-health study platform uses a citizen identification system called Bank-ID (Eaton et al., 2017). Bank-ID is connected to the individual mobile phone number and is regarded as a way of identification that provides the same level of security as a passport or a driver’s license, with the difference that the issuing bank guarantees the individual’s identity. This enables the participant to enroll in the study and sign the informed consent electronically, with their identity being authenticated by a third party.
Platform functionality
The platform provides a basis for time-efficient studies through offering functions for fully automated study information and registration of informed consent, but also randomization of participants, distribution of control interventions and screening surveys at baseline and follow-up. Aspects of RCTs that otherwise require extensive administrative effort, such as handling of informed consent, can be done automatically. In summary, the E-health study platform can be used to perform a research study and make sure that the same individual provides data at baseline and follow-up, even though the personal identity of this individual is unknown to the research team. This provides an ecologically valid way of evaluating interventions such as myCompass, which is not a health care intervention per se and where the idea is that the participant should be able to access the intervention without having to be enrolled as a patient in the health care system.
Evaluation of myCompass
Participants
Participants were recruited through print advertisements (sent out to Karolinska Institutet and major Swedish newspapers), social media (e.g., Facebook, Twitter, blogs) and information in regular health care services. The study was also presented in live morning news in national TV (December 15, 2015). About half of participants signed up within a week of the appearance on TV, indicating that this was the most important recruitment tool. Individuals interested in participating signed up through the E-health study platform and completed the informed consent form. During the registration period (December 2015–January 2017), 1207 individuals enrolled and consented to screening. Out of those, 1172 provided complete screening surveys (i.e., the Patient Health Questionnaire-9 [PHQ-9) and the Generalized Anxiety Disorder 7-item scale [GAD-7); described below). Out of these, 837 were deemed eligible to participate and were randomized into myCompass (n = 418) and the control intervention (n = 419). For a complete overview of the participant flow, see Fig. 1.
Fig. 1.
Flow-chart of participants. The flow-chart shows the flow of participants from initial enrollment to last follow-up survey
Inclusion criteria were; Swedish resident 18 years or older; having a valid e-mail address and reporting symptoms of mild to moderate depression and/or anxiety (i.e., a PHQ-9 score between 5 and 20 and/or a GAD-7 score between 5 and 15). Individuals who reported severe depression (≥20 on PHQ-9), or anxiety (≥15 on GAD-7) and/or suicidal thoughts were excluded. They received a standardized e-mail explaining why they were not included, with a strong recommendation to contact their M.D. or the emergency services.
Procedure
Individuals who fulfilled the eligibility criteria were randomized to myCompass or the control condition in a fully automated process through the E-health study platform. Participants randomized to the intervention condition were directed to a page at the platform where they were prompted to take part in myCompass during the following weeks. Identification and participant consent was initially done through e-mail and personal identity number/BankID. The personal identity number was used to check that participants were 18 years old or older. Following this checkpoint, all personal data was deleted.
In June 2016 (six months post study launch), five questions were added to the baseline survey in order to gather descriptive information on the participants (gender, living situation, children < 18 years, education and employment status). Nine questions were also added to follow-up survey 1 to gather usability data. The main reason for adding the questions was to explore possible explanations for the high drop-out rates, that were becoming evident at this point. In total, 157 participants (intervention n = 78; control n = 79) responded to the extended base line survey and 72 participants (intervention n = 39; control n = 33) responded to the extended follow up 1-survey. The questions were added after the majority of participants finished the intervention, the results thus stem from a smaller sub-group of participants. Of those who completed the extended baseline survey, most participants were female (63 and 66% in the myCompass group and controls, respectively), were employed more than 50% (55 and 70% in the myCompass group and controls, respectively) and had post-secondary education (63 and 68% in the myCompass group and controls, respectively), see Appendix Table 5.
Baseline- and outcome measures
Participants completed two self-report measures online at baseline; post-intervention (seven weeks after inclusion), and at follow-up (19 weeks after the inclusion). Both measures were administered through the E-health study platform.
The Patient Health Questionnaire-9 (PHQ-9; [35] is a 9-item measure that assesses degree of depressive symptoms. Items are scored between 0 (=not at all) to 3 (= nearly every day), with a total score of 27. A score between 0 and 4 indicates no depression; 5–9 minimal symptoms; 10–14 minor depression; 15–19 moderately severe major depression, and 20–27 severe major depression. PHQ-9 has demonstrated satisfactory psychometric properties with adequate internal consistency (Cronbach’s α ranging between .86–.89) and test-retest reliability (r = .84) [36].
The Generalized Anxiety Disorder 7-item scale (GAD-7; [37] is a seven-item measure that assesses levels of general anxiety disorder. GAD-7 is widely used as a general measure of anxiety, particularly in Swedish internet-based trials. Using GAD-7 makes comparisons to other, similar trials easier. Items are scored between 0 (= not difficult at all) to 3 (= extremely difficult), with a total score of 21. A score between 0 and 4 indicates no anxiety; 5–9 mild anxiety; 10–14 moderate anxiety; and 15–21 severe anxiety. GAD-7 has demonstrated satisfactory psychometric properties, with an internal consistency of α = .92, and test-retest reliability of r = .83 [36].
Interventions
myCompass
The myCompass is a completely automated self-guided program without therapist contact. The program contains 12 different modules targeting common mental health problems (e.g. anxiety, sleep disturbances, depression and stress). The modules are based on principles from cognitive behavior therapy, interpersonal psychotherapy, problem solving therapy and positive psychology. The participant can choose what modules to work with or opt for a tailored program based on responses in an online self-monitor questionnaires completed at registration. Each questionnaire consists of several questions on a certain topic such as sleep deprivation or anxiety. Ratings are made on a 10-point scale, ranging from 0 (= not at all) to 10 (= extremely) on questions such as “how stressed have you been feeling”; “how depressed have you been feeling” (during the past few weeks). The system is dynamic by allowing participants to re-take the questionnaire at any time point and thus the recommended modules can change. The modules consist of text material, images and homework assignments on specific topics (e.g., “setting smart goals”, “solving problems”; “sleeping well”).
myCompass also contains self-monitoring functions allowing participants to record their symptoms and track changes over time. The program uses email reminders, quotes sent as text messages, other participants’ stories and links to further reading about mental health (for a detailed description of the Australian version of myCompass, see Proudfoot, Clarke [31], Harrison, Proudfoot [32]). In the current study, participants logged into myCompass through a web browser (on computer, tablet or mobile phone); no separate application for tablets/mobile phones were constructed.
Control condition
Participants randomized to the control condition received a standardized program of one email per week during seven weeks in total, encompassing information on depression, stress and anxiety. In contrast to the intervention, the control condition did not contain any information on specific strategies to handle ones’ mental health issues (i.e. there was no individualization of the content). The information used was based on the control condition in the Australian evaluation of myCompass but was complemented with information relevant to the Swedish context, such as references to Swedish organizations and health care services, as well as activities relevant to Sweden e.g. “surfing” was replaced by “skiing”. The information was intended to take approximately 10 min to read on a computer. Participants in the control group were offered the myCompass intervention when the control intervention was finished.
Statistical analyses
Statistical analyses were conducted using XLSTAT 2018.3 (Build 50,965). Repeated Measures ANOVAs were performed to calculate between-group differences across time among completers (i.e., participants who completed the survey at baseline and both follow-ups). Such per-protocol analyses are associated with the risk of overestimating results, and with many possible confounding variables, but interpreted with caution it could provide the opportunity to detect the maximum possible benefit of the intervention. Linear mixed models (LMM) were used to adhere to the principle of intention to treat, allowing us to analyze the results using the complete data set for all participants (i.e., also including those who had only provided baseline data), see for example Hesser [38] for a discussion on the use of LMMs in randomized experiments. Maximum likelihood was used to handle missing data, under the assumption that data was missing at random. Active users (i.e., those who completed more than five out of 19 self-monitoring questionnaires and commenced at least one module in myCompass) were analyzed separately with both ANOVAs and linear mixed models for a more profound understanding of the results.
Results
Results are divided into: (1) adherence (also including number of self-monitor questions and modules completed in myCompass); (2) Repeated measures ANOVA’s for “completers”, and (3) mixed model outcomes for all included participants.
Adherence
Table 1 summarizes adherence to the myCompass intervention (i.e., as indicated by number of log-ins; number of commenced and completed modules; number of completed self-monitoring questions and completed follow-up measures) in all users and the smaller subgroup of “active users”. It also contains data on completion on follow-up surveys in the myCompass and control group. The results demonstrate that active users (n = 35) logged into myCompass more often than the larger group of all users (average number of log-ins 29.9 and 5.7, respectively); commenced and completed a larger number of modules (2.7 versus .6 and 1.8 versus .2), and self-monitor questions (97.6 versus 20.0). Regarding completion of the first and second follow-up survey, figures were equal and markedly higher among myCompass active users and participants in the control group, compared with the larger group of myCompass users.
Table 1.
Adherence to myCompass
|
myCompass
all users (n = 418) |
myCompass
active usersa (n = 35) |
Control group (n = 419) |
|||
|---|---|---|---|---|---|
| Avg. | SD | Avg. | SD | ||
| Avg. number of log-ins | 5.7 | 15.8 | 29.9 | 38.2 | – |
| Avg. number of modules commenced | 0.6 | 1.0 | 2.7 | 1.7 | – |
| Avg. number of modules completed | 0.2 | 0.7 | 1.8 | 1.5 | – |
| Number of myCompass self-monitor questions | 20.0 | 61.0 | 97.6 | 158.8 | – |
| Avg. number of days from registration to last log-in | 15.6 | 43.5 | 84.5 | 96.0 | – |
| Completed first follow-up survey | 19% | 49% | 48% | ||
| Completed second follow-up survey | 13% | 40% | 37% | ||
Note. In the myCompass intervention, modules target mental health problems (e.g., anxiety, sleep disturbances, depression and stress)
aActive users are defined as participants filling out > 5 self-monitor questionnaires and commencing at least one module
Figure 2 illustrates total number of log-ins to myCompass, demonstrating an uneven pattern. In total, 41% of participants never logged in after registering and less than 50% of participants logged in ten times or more.
Fig. 2.
Number of log-ins to myCompass. Displays the number of participants who logged in to myCompass, and how frequently
Repeated measures ANOVA’s for completers
Table 2 demonstrates between-group differences across the three assessment points (i.e., at baseline and follow-ups) for “completers” in the two groups of myCompass users (all users and active users) and controls. Among controls, there were significant improvements on both outcome measures (i.e., PHQ-9 and GAD-7), with scores decreasing from baseline to second follow-up. There was no significant difference between groups, however.
Table 2.
Repeated measures ANOVA for three subgroups, only including completersa
| Control group (n = 125) |
myCompass
all users (N = 28) |
myCompass
active users (n = 9) |
|||||||
|---|---|---|---|---|---|---|---|---|---|
| Score | 95% CI | p-value | Diff. from ctrl | 95% CI | p-value | Diff from ctrl | 95% CI | p-value | |
| PHQ-9 | |||||||||
| Pre-treatment | 10.36 | [9.58, 11.15] | – | 0.71 | [−1.12, 2.54] | 0.447 | 1.42 | [−1.61, 4.45] | .359 |
| Post-t change | −3.11 | [−3.86, − 2.37] | .000* | 0.11 | [− 1.36, 1.86] | 0.900 | 0.11 | [−2.77, 2.99] | .939 |
| 19 week change | −3.51 | [−4.26, − 2.77] | .000* | −0.02 | [−1.77, 1.73] | 0.979 | −1.93 | [−4.81, − 0.95] | .188 |
| GAD-7 | |||||||||
| Pre-treatment | 8.88 | [8.24, 9.54] | – | 0.08 | [−1.45, 1.60] | 0.922 | −0.11 | [−2.63, 2.41] | .932 |
| Post-t change | −2.34 | [−3.04, −1.65] | .000* | 0.49 | [−1.14, 2.11] | 0.556 | 0.90 | [−1.79, 3.58] | .511 |
| 19 week change | −2.36 | [−3.06, −1.66] | .000* | 1.00 | [−0.62, 2.63] | 0.226 | 0.58 | [−2.10, −3.27] | .670 |
Note. PHQ-9 The Patient Health Questionnaire-9. GAD-7 The Generalized Anxiety Disorder 7-item scale
aCompleters were defined as participants who completed all three surveys (at baseline and follow-ups)
*indicates a significant result at the 0.05 level
Mixed model outcomes
Table 3 demonstrates results for mixed models of all participants in the myCompass group and the control group on both outcome measures. The results demonstrate a significant effect of time (i.e., scores on both measures decrease in both follow-up measures), however no significant effects of group. Table 4 also demonstrates results for mixed models of all participants in both subgroups of myCompass users (i.e., all users and active users) and the control group, on both outcome measures. Again, results demonstrate a significant effect of time, however no significant effects of group.
Table 3.
Mixed models all users
| (N = 837) | Score | 95% CI | p-value | N |
|---|---|---|---|---|
| PHQ-9 | ||||
| Pre-treatment | 10.37 | [9.96, 10.78] | – | – |
| Post-t change | −2.53 | [−3.00, − 2.06] | .000* | – |
| 19 week change | − 2.97 | [− 3.50, − 2.44] | .000* | – |
| Control | 0.00 | – | – | 419 |
| myCompass | 0.27 | [−0.30, 0.83] | .353 | 418 |
| GAD-7 | ||||
| Pre-treatment | 9.05 | [8.73, 9.39] | – | – |
| Post-t change | −2.03 | [−2.45, −1.62] | .000* | – |
| 19 week change | −2.10 | [−2.56, − 1.63] | .000* | – |
| Control | 0.00 | – | – | 419 |
| myCompass | 0.24 | [−0.20, 0.67] | .280 | 418 |
Note. PHQ-9 The Patient Health Questionnaire-9. GAD-7 The Generalized Anxiety Disorder 7-item scale
Table 4.
Mixed models comparing controls, myCompass all users and my Compass active users
| (N = 837) | Score | 95% CI | p-value | N |
|---|---|---|---|---|
| PHQ-9 | ||||
| Pre-treatment | 10.37 | [9.96, 10.78] | – | – |
| Post-t change | −2.54 | [−3.01, − 2.07] | .000* | – |
| 19 week change | −2.98 | [− 3.51, − 2.45] | .000* | – |
| Control | 0.00 | 419 | ||
| myCompass | 0.22 | [−0.36, − 0.80] | .449 | 383 |
| Active users | 0.65 | [−0.71, −2.01] | .350 | 35 |
| GAD-7 | ||||
| Pre-treatment | 9.05 | [8.73, 9.37] | – | – |
| Post-t change | −2.04 | [−2.45, −1.62] | .000* | – |
| 19 week change | −2.10 | [−2.56, − 1.63] | .000* | – |
| Control | 0.00 | – | – | 419 |
| myCompass | 0.24 | [−0.21, 0.69] | .300 | 383 |
| Active users | 0.26 | [−0.78, 1.30] | .625 | 35 |
Note. PHQ-9 The Patient Health Questionnaire-9. GAD-7 The Generalized Anxiety Disorder 7-item scale
Discussion
This paper aimed to (i) share our experience of the E-health study platform (i.e., regarding security aspects, functionality) to which the myCompass intervention was linked, and (ii) report findings from the empirical evaluation of myCompass (i.e., prerequisites, execution, study outcomes), in a community sample. The E-health study platform proved to be an efficient tool to streamline study administration. The myCompass intervention rendered initial interest among a relatively high number of individuals, however failed to maintain engagement of those enrolled. A smaller subgroup of active users had a markedly higher activity, however their treatment results were not significantly better than those of controls.
The E-health study platform
The E-health study platform enabled the myCompass study to be delivered with a minimum of involvement from the study team. Randomized controlled trials are normally costly and require a rather extensive administrative effort, even for rather small trials. Research studies are required to be operated in a rigorous manner, demanding researchers to pay great attention to details regarding participant integrity and data security. The E-health study platform enables randomization, informed consent and evaluation to be administered in a fully automated manner, which minimizes the risks for omissions related to the human factor. This set up is rather unique and could simplify investigations of various fully automated interventions to a low cost. As mentioned earlier, this could ideally facilitate the spread and evaluation of evidence based interventions to an almost indefinite number of patients.
Health care providers struggle to offer services targeting mental health problems that are effective, easily accessible and cost-efficient. The combined approach using the E-health study platform and interventions like myCompass could prove to be one valuable approach, given that it is completely automated and easily accessible to anyone with an internet connection. However, some words of caution regarding the E-health study platform should be noted. First, while human errors could threaten the validity of research studies, human contact could make participants more willing to participate, even if that contact is limited to data collection and the handling of informed consent. Second, the E-health study platform is separate from the myCompass platform, which required participants to log on to two separate platforms. This could have deterred some participants from remaining in the study. With time, the transitions between different platforms will likely be smoother, as the platforms improve technically.
Empirical evaluation of myCompass
Despite the potential possibilities of self-guided mental health interventions [30], and the positive findings from the developer led evaluation of myCompass [31], the aim to transfer myCompass to a Swedish context and replicate the original empirical evaluation did not fully succeed. The study rendered substantial interest initially with 1207 individuals enrolling, but it generally failed to maintain the engagement of those who enrolled, especially in the intervention group. Few participants logged into myCompass more than once or twice, and 41% of enrolled participants never logged in. This indicates that many participants were undecided regarding the intervention and their own participation after they had enrolled, but before they had commenced the program. Participants in both groups significantly lowered their symptoms of depression and anxiety, but given the large number of dropouts, no firm conclusions can be made regarding the effectiveness of the program. With the caveat of the small study samples in mind, our findings do indicate that there was a lack of significant difference between myCompass and the active control. A smaller subgroup of active users (n = 35) had a markedly higher activity in the program, however their treatment results were not significantly better than those of the control group. It is worth noting that even among “active users”, there was a relatively modest use of myCompass modules overall (an average of 1.8 modules completed). Interestingly, the control condition completed a higher number of follow-up measures than the intervention group, possibly due to a ‘questionnaire fatigue’ experienced by the intervention group who were required to answer a number of questions also within the myCompass platform. For these reasons, it is impossible to draw any firm conclusions on the effectiveness of the intervention per se.
It is a known challenge that digital interventions without therapist contact have higher drop-out rates as well as small effect sizes [13, 26]. Also, transdiagnostic interventions may be more difficult to implement compared to diagnosis specific interventions, since they target a wider range of problems, and the participant thus may have to acquire more skills relative to programs focusing on a narrower problem area. While such factors may have affected the outcome of this study, the question remains why there were such marked differences compared to the original Australian study. While the mean number of completed modules in the Australian version was 1.6, it was 0.2 in Sweden, the participants logged in on average 14.7 times in Australia vs. 5.7 times in Sweden, attrition rate 56.7% in Australia vs. 87% in Sweden, and while the outcome measures were different, results in Australia were clearly superior to those in Sweden.
Information about the program was widespread, and given the number of responses on the website, the public was aware of its existence. Those who self-assessed their symptoms did report mental health problems – sometimes more severe than was intended for this particular program – indicating that we did reach a relevant target group. It is likely that the implementation of the Swedish version of myCompass did not fully take into account issues of transportability, however. Transportability usually refers to the process of implementing efficacious interventions to usual-care settings [39], and how differences in settings, organizations and culture could effect the efficiency of an intervention. It is possible that the composition and design of the myCompass platform better suited an Australian audience, and some of the images used might have come across as foreign in a Swedish context. For example, some text messages to the Swedish participants contained inspirational quotes predominantly citing famous Australians. Moreover, the platform contained several links to Australian organizations, and a handful of words on the platform were not translated from English to Swedish. This could create a sense of alienation from the program, especially since there was no therapist to counter doubts raised by these types of flaws. It is likely that a more thorough processing of the myCompass content would have rendered greater adherence and better outcomes, especially since other self-guided interventions for similar problems have proven successful, e.g. the Wellbeing Course [40]. There is also a possibility that the user interface design of myCompass appeared somewhat outdated, given that it was designed approximately five years prior to the Swedish participants enrolled in the study. Also, the technical development has meanwhile gravitated towards mobile phone applications, rather than web-based solutions, which might have exacerbated any sense of myCompass being outdated. On a more speculatively note, Sweden has been rated as the second most digitally advanced country in the world, with Australia on eleventh place [41]. This likely increases the bar for what is considered to be acceptable in terms of technology and interface design for the participants. A somewhat outdated platform, with some issues regarding cultural aptitude, and a slightly more tech-savvy audience might thus together explain some of the results.
While the results from the Swedish myCompass trial are substantially poorer than the Australian counterpart, the level of adherence is similar to other self-guided interventions where it is a common finding that 90% of participants withdraw after two sessions [30]. This makes it difficult to draw empirically based conclusions on the effectiveness of these types of programs. Given the far-reaching dissemination of myCompass, it could perhaps be regarded as a tailored public health intervention for prevention of mental health symptoms (i.e., rather than providing treatment per se). Public health interventions are generally more difficult to evaluate, not least since they target a whole population rather than individuals, see for example Jackson et al. (2004) for a discussion on public health methodology. While the measurable effect of myCompass is small, it could still be one of several factors raising awareness and promoting mental health change. There is also a possibility that individuals who enroll in this type of intervention also seek regular mental health care or attend support groups; people may try out different alternatives before committing to one. For example, therapist-guided internet-interventions were, and are, offered through regular health care for several of the issues targeted in this study, e.g. insomnia [42] and depression [43]. The availability of similar self-guided mobile phone applications for promoting mental health like Shimdi [44], and English language applications like Woebot [45] should also be mentioned. It is likely that some participants moved on to other alternatives. This relative abundance of treatment alternatives is in line with the proposed apotecharies of digital care [24], but it could have the downside of pushing clients into a constant search for an envisioned perfect treatment option.
Conclusion
Several valuable lessons can be drawn from the implementation and evaluation of myCompass in a Swedish context. The number of participants initially registering demonstrates that self-guided interventions could constitute viable alternatives for many people seeking treatment. The number of people who remain active in the intervention is much more limited than the number of individuals initially signing up, however. Another lesson is that these types of interventions need to be carefully transported to new contexts and languages to appear credible. Interventions must not necessarily be “home-grown”, but a thorough processing involving pilot trials is likely needed before large-scale studies are conducted. Finally, the E-health study was a highly useful tool, indicating that it has the potential to facilitate any intervention study of this type. A challenge for the future will be to stay in par with rapidly developing techniques to provide secure and efficient internet-based interventions for various patient groups.
Acknowledgements
The research team whishes to offer their special thanks to Lotta Jernström, Marit Grönberg Eskel and Ing-Marie Wieselgren at SALAR for their important assistance during this study, as well as the Black Dog Institute in Australia for their invaluable cooperation.
Funding
This study was financed by the Swedish Association of Local Authorities and Regions (SALAR). SALAR is an association consisting of representatives from all municipalities and regions in Sweden. The funder had no influence on the collection, analysis, interpretation of data or in writing this manuscript. The authors of the paper independently created the research design. The funding body got the opportunity to confirm the research design before the start of the project.
Availability of data and materials
The data used in the myCompass study are available from the corresponding author on reasonable request. The data are presented in an anonymous way.
Abbreviations
- CBT
Cognitive behavioral therapy
- CG
Control group
- EOI
Expression of interest
- GAD-7
Generalized Anxiety Disorder 7-item scale
- ICBT
Internet-delivered cognitive behavioral therapy
- LMM
Linear mixed models
- PHQ-9
Patient Health Questionnaire–9
Appendix
Table 5.
Descriptive information for the subgroup of participants registered post May 2016, divided into MyCompass and controls
|
My Compass
(n = 78) |
Controls (n = 79) |
|
|---|---|---|
| Gender | ||
| Female | 49 (63%) | 52 (66%) |
| Male | 27 (35%) | 23 (29%) |
| Other/blank | 2 (3%) | 4 (5%) |
| Employmenta | ||
| Employed more than 50% | 43 (55%) | 55 (70%) |
| Employed less than 50% | 14 (18%) | 10 (13%) |
| Retired | 21 (27%) | 14 (18%) |
| Education | ||
| No formal education | 7 (9%) | 2 (3%) |
| Elementary school | 5 (6%) | 0 (0%) |
| Secondary school | 10 (13%) | 22 (28%) |
| Post-secondary education (e.g. university education) | 49 (63%) | 54 (68%) |
Note. aEmployment includes any participation in the labor force such as working or studying
Authors’ contributions
AN participated in the manuscript writing and statistical analysis of this paper. JL participated in the research design, statistical analysis and manuscript writing of this article. EÖ participated in the data collection and manuscript writing of this paper. JK participated in the research design, data collection and manuscript writing of this paper. CH participated in the launch of the platform, communication strategy, ethical application, monitoring of the study, analysis and interpretation of the data. KS participated in the manuscript writing and statistical analysis of this paper. All authors read and approved the final manuscript.
Ethics approval and consent to participate
This study was approved by the Regional Ethical Review Board in Stockholm, with the protocol number 2015/1268–31/2 + 2016/88. All participants filled out a statement regarding informed consent to participate online, distributed via the E-health platform described above.
Consent for publication
N/A
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Anders Nilsson, Email: anders.nilsson.2@ki.se.
Karolina Sörman, Email: Karolina.sorman@ki.se.
Josefin Klingvall, Email: Josefin.klingvall@lumell.se.
Emma Ovelius, Email: Emma.ovelius@lumell.se.
Jonas Lundberg, Email: Jonas.lundberg@lumell.se.
Clara Hellner, Email: Clara.hellner@ki.se.
References
- 1.Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the global burden of disease study 2010. Lancet. 2013;382(9904):1575–1586. doi: 10.1016/S0140-6736(13)61611-6. [DOI] [PubMed] [Google Scholar]
- 2.Steel Z, Marnane C, Iranpour C, Chey T, Jackson JW, Patel V, et al. The global prevalence of common mental disorders: a systematic review and meta-analysis 1980–2013. Int J Epidemiol. 2014;43(2):476–493. doi: 10.1093/ije/dyu038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ferrari AJ, Charlson FJ, Norman RE, Patten SB, Freedman G, Murray CJ, et al. Burden of depressive disorders by country, sex, age, and year: findings from the global burden of disease study 2010. PLoS Med. 2013;10(11):e1001547. doi: 10.1371/journal.pmed.1001547. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.World Health Organization. Comprehensive mental health action plan 2013–2020. Geneva: World Health Organization. p. 2013.
- 5.Henderson M, Harvey S, Øverland S, Mykletun A, Hotopf M. Work and common psychiatric disorders. J R Soc Med. 2011;104(5):198–207. doi: 10.1258/jrsm.2011.100231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Wang PS, Angermeyer M, Borges G, Bruffaerts R, Chiu WT, De Girolamo G, et al. Delay and failure in treatment seeking after first onset of mental disorders in the World Health Organization's world mental health survey initiative. World Psychiatry. 2007;6(3):177. [PMC free article] [PubMed] [Google Scholar]
- 7.Golberstein E, Eisenberg D, Gollust SE. Perceived stigma and mental health care seeking. Psychiatr Serv. 2008;59(4):392–399. doi: 10.1176/ps.2008.59.4.392. [DOI] [PubMed] [Google Scholar]
- 8.Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC psychiatry. 2010;10(1):113. doi: 10.1186/1471-244X-10-113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Mojtabai R, Olfson M, Sampson NA, Jin R, Druss B, Wang PS, et al. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med. 2011;41(8):1751–1761. doi: 10.1017/S0033291710002291. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Tito N, Dear BF, Staples LG, Bennett-Levy J, Klein B, Rapee RM, Andersson G, Purtell C, Bezuidenhout G, Nielssen OB. The first 30 months of the MindSpot Clinic: Evaluation of a national e-mental health service against project objectives. Aust N Z J Psychiatry. 2017;51(12):1227-39. [DOI] [PubMed]
- 11.Barak A, Klein B, Proudfoot JG. Defining internet-supported therapeutic interventions. Ann Behav Med. 2009;38(1):4–17. doi: 10.1007/s12160-009-9130-7. [DOI] [PubMed] [Google Scholar]
- 12.Andersson G. Using the internet to provide cognitive behaviour therapy. Behav Res Ther. 2009;47(3):175–180. doi: 10.1016/j.brat.2009.01.010. [DOI] [PubMed] [Google Scholar]
- 13.Andersson G. Internet-delivered psychological treatments. Annu Rev Clin Psychol. 2016;12:157–179. doi: 10.1146/annurev-clinpsy-021815-093006. [DOI] [PubMed] [Google Scholar]
- 14.Cunningham JA, Wild TC, Cordingley J, Van Mierlo T, Humphreys K. A randomized controlled trial of an internet-based intervention for alcohol abusers. Addiction. 2009;104(12):2023–2032. doi: 10.1111/j.1360-0443.2009.02726.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Titov N, Andrews G, Johnston L, Robinson E, Spence J. Transdiagnostic internet treatment for anxiety disorders: a randomized controlled trial. Behav Res Ther. 2010;48(9):890–899. doi: 10.1016/j.brat.2010.05.014. [DOI] [PubMed] [Google Scholar]
- 16.Cunningham JA, Gulliver A, Farrer L, Bennett K, Carron-Arthur B. Internet interventions for mental health and addictions: current findings and future directions. Curr Psychiatry Rep. 2014;16(12):521. doi: 10.1007/s11920-014-0521-5. [DOI] [PubMed] [Google Scholar]
- 17.Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. 2014;13(3):288–295. doi: 10.1002/wps.20151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Cuijpers P, van Straten A, Andersson G. Internet-administered cognitive behavior therapy for health problems: a systematic review. J Behav Med. 2008;31(2):169–177. doi: 10.1007/s10865-007-9144-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Griffiths KM, Christensen H. Review of randomised controlled trials of internet interventions for mental disorders and related conditions. Clin Psychol. 2006;10(1):16–29. doi: 10.1080/13284200500378696. [DOI] [Google Scholar]
- 20.Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007;37(03):319–328. doi: 10.1017/S0033291706008944. [DOI] [PubMed] [Google Scholar]
- 21.Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med. 2010;40(12):1943–1957. doi: 10.1017/S0033291710000772. [DOI] [PubMed] [Google Scholar]
- 22.Karyotaki E, Ebert DD, Donkin L, Riper H, Twisk J, Burger S, et al. Does guided internet-based interventions result in clinically relevant changes for patients with depression? An individual participant data meta-analysis. Clin Psychol Rev. 2018;63:80–92. doi: 10.1016/j.cpr.2018.06.007. [DOI] [PubMed] [Google Scholar]
- 23.Rozental A, Magnusson K, Boettcher J, Andersson G, Carlbring P. For better or worse: an individual patient data meta-analysis of deterioration among participants receiving internet-based cognitive behavior therapy. J Consult Clin Psychol. 2017;85(2):160. doi: 10.1037/ccp0000158. [DOI] [PubMed] [Google Scholar]
- 24.Muñoz RF, Chavira DA, Himle JA, Koerner K, Muroff J, Reynolds J, et al. Digital apothecaries: a vision for making health care interventions accessible worldwide. mHealth. 2018;4(6):18. doi: 10.21037/mhealth.2018.05.04. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Swartz L, Noell J, Schroeder S, Ary D. A randomised control study of a fully automated internet based smoking cessation programme. Tob Control. 2006;15(1):7–12. doi: 10.1136/tc.2003.006189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on internet-based mental health interventions—a systematic review. Internet Interv. 2014;1(4):205–215. doi: 10.1016/j.invent.2014.08.003. [DOI] [Google Scholar]
- 27.Twomey C, O’Reilly G. Effectiveness of a freely available computerised cognitive behavioural therapy programme (MoodGYM) for depression: meta-analysis. Aust N Z J Psychiatry. 2017;51(3):260–269. doi: 10.1177/0004867416656258. [DOI] [PubMed] [Google Scholar]
- 28.Klein B, Meyer D, Austin DW, Kyrios M. Anxiety online—a virtual clinic: preliminary outcomes following completion of five fully automated treatment programs for anxiety disorders and symptoms. J Med Internet Res. 2011;13(4):e89. doi: 10.2196/jmir.1918. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e11. doi: 10.2196/jmir.7.1.e11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Titov N, Dear BF, Johnston L, Lorian C, Zou J, Wootton B, et al. Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: randomised controlled trial. PLoS One. 2013;8(7):e62873. doi: 10.1371/journal.pone.0062873. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Proudfoot J, Clarke J, Birch M-R, Whitton AE, Parker G, Manicavasagar V, et al. Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial. BMC psychiatry. 2013;13(1):312. doi: 10.1186/1471-244X-13-312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Harrison V, Proudfoot J, Wee PP, Parker G, Pavlovic DH, Manicavasagar V. Mobile mental health: review of the emerging field and proof of concept study. J Ment Health. 2011;20(6):509–524. doi: 10.3109/09638237.2011.608746. [DOI] [PubMed] [Google Scholar]
- 33.Organisation for Economic Co-operation and Development. Sick on the job?: myths and realities about mental health and work. Paris: OECD publishing; 2012.
- 34.Solomon D, Proudfoot J, Clarke J, Christensen H. E-CBT (myCompass), antidepressant medication, and face-to-face psychological treatment for depression in Australia: a cost-effectiveness comparison. J Med Internet Res. 2015;17(11):e255. doi: 10.2196/jmir.4207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Kroenke K, Spitzer RL, Williams JBW. The PHQ-9. J Gen Intern Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Kroenke K, Spitzer RL, Williams JB, Löwe B. The patient health questionnaire somatic, anxiety, and depressive symptom scales: a systematic review. Gen Hosp Psychiatry. 2010;32(4):345–359. doi: 10.1016/j.genhosppsych.2010.03.006. [DOI] [PubMed] [Google Scholar]
- 37.Spitzer RL, Kroenke K, Williams JW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–1097. doi: 10.1001/archinte.166.10.1092. [DOI] [PubMed] [Google Scholar]
- 38.Hesser H. Modeling individual differences in randomized experiments using growth models: recommendations for design, statistical analysis and reporting of results of internet interventions. Internet Interv. 2015;2(2):110–120. doi: 10.1016/j.invent.2015.02.003. [DOI] [Google Scholar]
- 39.Schoenwald SK, Hoagwood K. Effectiveness, transportability, and dissemination of interventions: what matters when? Psychiatr Serv. 2001;52(9):1190–1197. doi: 10.1176/appi.ps.52.9.1190. [DOI] [PubMed] [Google Scholar]
- 40.Titov N, Dear BF, Johnston L, McEvoy PM, Wootton B, Terides MD, et al. Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: a 12-month follow-up of a randomised controlled trial. PLoS One. 2014;9(2):e89591. doi: 10.1371/journal.pone.0089591. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Chakravorti B, Chaturvedi R. Digital planet 2017. How competitiveness and trust in digital economies vary across the world. The Fletcher School Tufts University https://sites.tufts.edu/digitalplanet/files/2017/05/Digital_Planet_2017_FINAL.pdf. 2017.
- 42.Kaldo V, Jernelöv S, Blom K, Ljótsson B, Brodin M, Jörgensen M, et al. Guided internet cognitive behavioral therapy for insomnia compared to a control treatment–a randomized trial. Behav Res Ther. 2015;71:90–100. doi: 10.1016/j.brat.2015.06.001. [DOI] [PubMed] [Google Scholar]
- 43.Hedman E, Ljótsson B, Kaldo V, Hesser H, El Alaoui S, Kraepelien M, et al. Effectiveness of internet-based cognitive behaviour therapy for depression in routine psychiatric care. J Affect Disord. 2014;155:49–58. doi: 10.1016/j.jad.2013.10.023. [DOI] [PubMed] [Google Scholar]
- 44.Ly KH, Ly A-M, Andersson G. A fully automated conversational agent for promoting mental well-being: a pilot RCT using mixed methods. Internet Interv. 2017;10:39–46. doi: 10.1016/j.invent.2017.10.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Fitzpatrick KK, Darcy A, Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR mental health. 2017;4(2):e19. doi: 10.2196/mental.7785. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data used in the myCompass study are available from the corresponding author on reasonable request. The data are presented in an anonymous way.