Effectiveness of online interventions in preventing depression: a protocol for systematic review and meta-analysis of randomised controlled trials

Abstract

Introduction

Although evidence exists for the efficacy of psychosocial interventions in preventing depression, little is known about its prevention through online interventions. The objective of this study is to conduct a systematic review and meta-analysis of randomised controlled trials assessing the effectiveness of online interventions in preventing depression in heterogeneous populations.

Methods and analysis

We will conduct a systematic review and meta-analysis of randomised controlled trials that will be identified through searches of PubMed, PsycINFO, WOS, Scopus, OpenGrey, Cochrane Central Register of Controlled Trials, ClinicalTrials.gov and Australia New Zealand Clinical Trials Register. We will also search the reference lists provided in relevant studies and reviews. Experts in the field will be contacted to obtain more references. Two independent reviewers will assess the eligibility criteria of all articles, extract data and determine their risk of bias (Cochrane Collaboration Tool). Baseline depression will be required to have been discarded through standardised interviews or validated self-reports with standard cut-off points. The outcomes will be the incidence of new cases of depression and/or the reduction of depressive symptoms as measured by validated instruments. Pooled standardised mean differences will be calculated using random-effect models. Heterogeneity and publication bias will be estimated. Predefined sensitivity and subgroup analyses will be performed. If heterogeneity is relevant, random-effect meta-regression will be performed.

Ethics and dissemination

The results will be disseminated through peer-reviewed publication and will be presented at a professional conference. Ethical assessment is not required as we will search and assess existing sources of literature.

Trial registration number

CRD42014014804; Results.

Strengths and limitations of this study.

• This is the first systematic review and meta-analysis of randomised controlled trials assessing the effectiveness of online interventions in preventing depression in a heterogeneous population.

• The study will be conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for reporting systematic reviews and meta-analysis to meet the highest scientific quality.

• The effect size, robustness and quality of evidence found in this meta-analysis will help determine whether depression can be prevented through online interventions.

• This study will have the limitations inherent to any systematic review and meta-analysis, such as the loss of information on outcome variables, or the assumption that the evaluation techniques are consistent across studies.

• The number and quality of the randomised controlled trials included, heterogeneity and occurrence of publication bias might limit the interpretation of results.

Introduction

Depression is a common, resource-consuming and disabling mental disorder that reduces life expectancy.¹ There are currently 322 million people with depression in the world.² The average lifetime and 12-month prevalence estimates of DSM-IV Major Depression Episode in high-income countries are 14.6% and 5.5%, respectively.³ In the last 10 years, the burden of major depression measured as years lived with disability (YLD) increased by 17.8%, ranking third in the world in disease burden³ and will rank first in high-income countries by 2030.⁴ In addition, depression is the primary cause of disability in the world attributable to mental and substance use disorders.⁵

Current treatments for depression show several constraints such as accessibility issues, limited efficacy or lack of adherence.^6–8 Even if it was possible to provide appropriate treatments to all persons affected by a depressive disorder, the effect on averting YLD would be limited because of the steady influx of new cases of depression.⁹ For all these reasons, the burden of depression can only be reduced by 20%–30%.¹⁰ Prevention may offer new possibilities to reduce the disease burden of depressive disorders.^{11 12}

Hundreds of randomised controlled trials (RCTs) and dozens of systematic reviews (SRs)/meta-analyses (MA) have been published on interventions to prevent depression.^13–17 A systematic review of SR/MA of psychological and/or educational interventions to prevent depression included 12 SR/MA (156 non-repeated trials and 56 158 participants) and found a small-moderate preventive effect.¹⁸ If preventive interventions reach a significant part of the population, even if this effect size is small, the impact on health, quality of life and healthcare costs could be relevant. From this point of view, scalability is crucial to prevention interventions. Solutions may leverage technological advances, such as mHealth-based counselling, computer and web-based resources.¹⁹

Interest in online prevention programmes for depression has increased substantially in recent years.^20–22 Online interventions offer some advantages over face-to-face interventions for both patients and the health system. Its advantages include greater intimacy, lower economical costs, the opportunity of joining the intervention at any time and place, easy access to a wider range of people (disabled population, rural areas, etc) and a reduction in the time of waiting, among others.^23–27

So far, three SR/MA on the effectiveness of online interventions in preventing mental disorders have been published.^28–30 To our knowledge, these previous reviews have some limitations. One²⁸ was focused on several mental disorders altogether (eating, anxiety, insomnia, post-traumatic stress, depression and common mental health disorders) and included only four trials on the prevention of depression. A limitation was the inclusion of some studies that only reported mean scores and did not clearly state that participants did not exceed clinical cut-offs at baseline, thus making it difficult to separate treatment from prevention. The other²⁹ was focused on the online cognitive behavioural therapy for subthreshold depression, which restricts its inference for that kind of psychotherapy and only for a type of prevention, indicated prevention; not addressed, thus, universal and selective prevention strategies. In addition, new RCTs on online interventions for the prevention of depression have been published.^31–35 The last SR/MA³⁰ found a small preventive effect of the eHealth interventions to prevent anxiety and depression; however, there were some exclusion criteria which limited their inferences: age (18–64 years), language (English), date (from 2000) and non-specific population (eg, postnatal or comorbid). Therefore, the objective of this study will be to conduct a systematic review and meta-analysis of RCTs assessing the effectiveness of online interventions in preventing depression in heterogeneous populations.

Methods and analysis

We followed Preferred Reporting Items for Systematic review and Meta-Analysis Protocols guidelines for reporting systematic reviews and meta-analysis protocols.³⁶ The protocol of this study has already been registered with the International Prospective Register of Systematic Reviews, (PROSPERO) on 20 November 2014 and was last updated on 23 November 2017 (registration number: CRD42014014804).

Eligibility criteria

The rationale for the inclusion and exclusion criteria outlined below is to obtain a comprehensive overview of the RCTs performed so far assessing the effectiveness of online interventions in preventing depression in different populations and settings.

Study design

We will focus on RCTs since this design provides more evidence on causality and is considered a gold standard for clinical trials.³⁷ Cluster randomised trials will only be included if there are at least two intervention and control sites and outcomes are reported adjusted for clustering effect. Controlled non-randomised clinical trials or before-after trials will be excluded.

Participants and exclusion of depression at baseline

Participants may have any sociodemographic (age, sex, etc) or clinical (healthy, chronic physical illness, etc) characteristic and all settings (community, schools, primary care, etc) will be considered. To make a clear distinction between the effectiveness of prevention interventions from that of treatments, baseline depression will be required to have been discarded through standardised interviews (eg, Composite International Diagnostic Interview, CIDI) or validated self-reports with standard cut-off points (eg, Beck Depression Inventory, BDI-II). In a preventive context, the most useful parameter of validity of a diagnostic instrument to discard depression at baseline is the ‘negative predictive value’ (NPV): probability of having a depressive disorder when the result of a diagnostic tool is negative. The NPV is influenced by three main parameters: cut-off selected, sensitivity associated with that cut-off and prevalence of depressive disorders in the reference population of the study. Higher sensitivity, lower cut-off and prevalence will increase the NPV and minimise false negatives. Structured standardised interviews generally have greater validity than symptom scales and, therefore, the former are preferable. However, structured standardised interviews tend to have greater specificity than sensitivity; therefore, false positives will be minimised at the expense of increasing false negatives. From this point of view, a symptoms scale with a diagnostic threshold associated with high sensitivity could guarantee as valid as a structured standardised interview, especially if the study is carried out on a reference population with a low prevalence of depressive disorders, as it is usual in prevention studies.

Type of interventions

We will only include RCTs assessing the effectiveness of psychosocial and/or educational since they share the same mechanism of action that facilitates changes in attitudes and behaviours and because most interventions to prevent depression are of this type. Educational interventions provide information sessions or fact sheets, whereas psychosocial interventions attempt to change how people think and behave by using a variety of strategies (eg, cognitive-behavioural or interpersonal). However, in real practice psychosocial and educational interventions can overlap, being difficult to distinguish them. Interventions must be accessible online and the study should include at least an internet-delivered intervention programme. If no online intervention is implemented in any of the experimental arms, the RCT will be excluded. Intervention arms where active pharmacological therapies are administered will also be excluded.

Comparators

Comparator groups could be ‘only assessments’, ‘no treatment’, ‘usual care’, ‘waiting list’ or any type of active control which has no effect on depression. All types of placebo (psychological or pill) will also be accepted as comparators. Comparator arms which intervention (psychological, physical or pharmacological) has been proven to be effective in preventing depression will also be excluded.

Outcomes

RCTs which primary or secondary outcomes were the incidence of new cases of depression and/or the reduction of depression symptoms will be included. Outcomes will be required to have been measured by standardised interviews or validated symptom scales. When more than a symptom scale has been used to measure outcomes in an RCT, the data from the highest validity scale will be employed. If the validation data of the scales, in the country and setting where the study was conducted, are not reported in the article, they will be searched in the literature and other sources. The parameters that will be used to select the scale of symptoms are higher Youden’s J statistic (J=sensitivity+specificity – 1), Cronbach alpha and intraclass correlation coefficient (test–retest) and sensitivity to change over time (yes/no/not available). For each trial, the scale of symptoms that provide more validation data and of higher quality will be chosen. If depression outcomes are measured together with other outcomes (eg, anxiety) and data are not provided separately, RCTs will be excluded.

Information sources and search strategy

A literature search of the following electronic databases will be carried out: PubMed, PsycINFO, WOS, Scopus and Cochrane Central Register of Controlled Trials. Search will be supplemented by searching for trial protocols on ClinicalTrials.gov and Australia New Zealand Clinical Trials Register. We will also examine OpenGrey (System for Information on Grey Literature in Europe), where grey literature is indexed. PROSPERO will be searched for ongoing or recently completed systematic reviews. To ensure literature saturation, we will also review reference lists from relevant systematic reviews and meta-analysis and those from the RCTs included in our SR/MA. In addition, expert authors will be contacted in order to identify missing articles in our search. Literature search strategies will be developed using medical subject headings and text words related to prevention, depression and internet intervention. No limits will be imposed on study publication language or publication date. The search will be updated toward the end of the review. A draft MEDLINE search strategy in PICOS format is included in online supplementary file. We will adapt the MEDLINE strategy to the syntax and subject headings of the other databases.

Study selection

The entire selection process will be conducted independently by two reviewers. After elimination of duplicate studies, all records will be reviewed. Based on their titles and abstracts, the studies that do not meet inclusion criteria will be ruled out. The full text of the studies selected as potentially relevant will be reviewed for further assessment. Any disagreements will be discussed and resolved by consensus or by a third independent reviewer, if necessary. We will seek additional information from corresponding authors when necessary to resolve questions about eligibility. We will record the reasons for excluding trials. The reviewers will not be blind to the journal titles or the study authors or institutions. Inter-agreement of the total selection will be assessed using kappa,³⁸ which can be interpreted as follows: <0.20 as poor, 0.21–0.40 as fair, 0.41–0.60 as moderate, as 0.61–0.80 as good and 0.81–1.00 as excellent.

Data extraction

Data extraction from each eligible study will be conducted independently by two reviewers. Any disagreement will be discussed and resolved by consensus or by a third independent reviewer. We will also contact authors to get incomplete or unclear information, where appropriate. Abstracted data will include author/year and country; setting, target population characteristics (age, sex, etc) and type of prevention (universal, selective or indicate); sample size (control/intervention); exclusion of depression criteria at baseline and validated instruments used; orientation and intervention type and intervention details in both experimental and control groups (type, modes of application, frequency, intensity and level of adherence); prevention depression outcomes and validated instruments used; and all follow-up provided from the RCTs. Whenever possible, we will use results from intention-to-treat analysis.

Risk of bias

The quality of the articles will be assessed using the six criteria of risk bias proposed by the Cochrane Collaboration tool: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data (eg, dropouts and withdrawals) and selective reporting. To manage the risk of bias as a quantitative variable in meta-regression, it will be assessed by assigning the zero points to low-risk criteria, one to unclear and two to high-risk criteria. Therefore, the highest risk of bias score will be 12 and the lowest 0. The risk of bias will be assessed independently by two reviewers. In case of disagreement, a third reviewer will be consulted. The inter-agreement will be rated using intraclass correlation coefficient.³⁸ The original study investigators will be contacted for more information, when necessary.

Assessment of publication bias

Publication bias will be evaluated by inspecting the funnel plot on the primary outcome measure and by Duval and Tweedie’s trim-and-fill procedure,³⁹ which yields an estimate of the effect size after adjusting for publication bias. The funnel plot is expected to be symmetric, equally dispersed on the general effect. If there are missing studies, the trim and fill procedure imputes these studies and adds them to the analysis. We will also perform Begg and Mazumdar rank correlation⁴⁰ and Egger’s test.⁴¹ If asymmetry is potentially caused by publication bias, we expect to see high SEs (small studies) associated with larger effect sizes.

Patient and public involvement

No patients or public will be involved in the study.

Statistical analysis

All statistical analyses will be performed using the Comprehensive Meta-Analysis (CMA) software package, V.2.2.021 and STATA-Release V.14.2. Standardised mean difference (SMD) will be used as effect size as most RCTs included in our meta-analysis are expected to report differences in symptoms of depression. For each study, we will first calculate the SMD by merging the SMD at different follow-up times into a single average estimate. We then will calculate the pooled SMD of all RCTs as well as its 95% CI. If some RCT only reports new cases of depression’ (incidence of depression), CMA will be used to obtain the equivalent SMD. Negative SMDs (between intervention and control group) will indicate a better outcome (reduction of depressive symptoms) in the intervention group. Following the interpretation proposed by Cohen for this effect size: −0.2 is small; −0.5 medium and −0.8 large.⁴² We will inflate the SEs of the nested comparisons in the same RCT following the suggestions of Cates.⁴³ A priori, we selected a random-effects model for our meta-analysis under the assumption that the RCTs to be included in our study will be performed in heterogeneous ‘populations’ that may differ from each other.⁴⁴

To test the heterogeneity of effect sizes, I² and its 95% CI will be calculated and expressed as percentages, where a value of 0%–40% might be unimportant heterogeneity, 30%–60% moderate, 50%–90% substantial and 75%–100% considerable.⁴⁴ We will also calculate the Cochran’s Q statistic and its p value.

We will perform the following sensitivity analyses: at first and last follow-up, using fixed effects and Hedges’g and excluding some RCTs from analysis (eg, those which cause the greatest increase in heterogeneity).

We will use a mixed-effects model for subgroup analyses based on a set of variables selected a priori, as follows: type of prevention (universal, selective or indicated), prevention of depression as primary or secondary outcome, type of outcome measure (symptoms scale vs standardised diagnostic interview), country, population age, setting (school, primary care, etc), comparator (waiting list, usual care, active control), intervention orientation (Cognitive Behavioural Therapy, other), intervention format, intervention guidance (guided or unguided), number of sessions or impacts, follow-up, level of usability or adherence (if it was measured), sample size and risk of bias.

Random-effect meta-regressions will be performed to investigate whether there are differences in effect sizes over time or according to the risk of bias. Normality of quantitative variables will be verified by the skewness and kurtosis normality test prior to inclusion in meta-regression analysis⁴⁵; transformations will be conducted, when appropriate, to get approximation to normality. If significant heterogeneity is observed, the covariables with a p<0.15 which were not removed from the model due to collinearity will be also included in meta-regression models. Risk of bias and sample size will be forced into meta-regression models to adjust for them, the latter only in case of detection of publication bias. SE and CIs will be calculated using the Knapp and Hartung method.⁴⁶ P values will be calculated using Higgins and Thompson⁴⁷ permutation test approach, taking into account multiplicity adjustment, when necessary. A normal probability plot of standardised shrunken residuals will be used to estimate the goodness of fit of meta-regression models.

The quality of evidence

The quality of evidence in the domains of risk of bias, consistency, directness, precision and publication bias will be assessed using the Grading of Recommendations Assessment, Development and Evaluation working group methodology.⁴⁸ Additional domains may be considered, where appropriate.

Ethics and dissemination

The results will be disseminated through peer-reviewed publication and will be presented at a professional conference. Ethical assessment is not required as we will search and assess existing sources of literature.