Dismantling, personalising and optimising internet cognitive–behavioural therapy for depression: a study protocol for individual participant data component network meta-analysis

Abstract

Introduction

Psychotherapy is a complex intervention, consisting of various components and being implemented flexibly in consideration of individual patient’s characteristics. It is then of utmost importance to know which of the various components or combinations thereof are more efficacious, what their specific effect sizes are and which types of patients may benefit more from different components or their combinations.

Methods and analysis

Internet-delivered cognitive–behavioural therapy (iCBT) offers a unique opportunity to systematically review and quantitatively disentangle the efficacy of various components because, unlike face-to-face cognitive–behavioural therapy, it allows identification of constituent components that are actually delivered to patients. We will systematically identify all randomised controlled trials that compared any form of iCBT against another form or a control intervention in the acute phase treatment of adult depression. We will apply component network meta-analysis (cNMA) to dismantle efficacy of individual components. We will use individual participant data in the cNMA to identify participant-level prognostic factors and effect modifiers for different components.

Ethics and dissemination

The investigators of the primary trials will have obtained ethical approval for the data used in the present study and for sharing the data, if this was necessary, according to local requirements and was not covered from the initial ethic assessment. Results from this study will be published in peer-reviewed journals and presented at relevant conferences.

PROSPERO registration number

CRD42018104683.

Strengths and limitations of this study.

• Internet-delivered cognitive–behavioural therapy will allow identification of constituent components used in each intervention and effectively delivered to participants.

• Component network meta-analysis (cNMA) will allow estimation of a specific incremental effect size for each component.

• Individual participant data cNMA (IPD-cNMA) will allow identification of prognostic factors and effect modifiers for different components.

• IPD-cNMA, while a powerful methodology, is limited by the availability of individual participant data, their quality and their comprehensiveness.

Introduction

Psychotherapy is a complex intervention, comprising multiple components in various combinations.¹ Psychotherapy may be even more complex in practice, as its implementation is variable and flexible even in research settings. It is generally believed, though seldom empirically demonstrated,² that efficacy of psychotherapy is moderated by individual patients’ characteristics.

Cognitive–behavioural therapy (CBT) is the most widely studied type of psychotherapy for depression. CBT, however, should be considered more as an umbrella term, because under the general label of CBT, we can find psychotherapies that include various combinations of many different components. Moreover, these are administered in a flexible manner, presumably in concordance with individual patient characteristics. Viewed from the clinicians’ and consumers’ points of view, it is then of utmost importance to determine which of the various components or combinations of components proposed for CBT are more efficacious, what their corresponding effect sizes are and which types of patients may benefit more from different components or their combinations.

Apart from a few exceptions known as dismantling studies,³ research into CBT has usually studied it as a package of various components. The complexity of psychotherapeutic interventions noted above at the levels of intervention and population applies to such research and adds much uncertainty to their analyses and interpretations.^{4 5} In traditional systematic reviews of psychotherapeutic interventions, identification of specificity in psychotherapy has proved extremely difficult.^{6 7} Detection of differences among treatments and treatment components would require extremely large samples.³ In addition, the influence of patient-level characteristics on the efficacy of the interventions could not be adequately explored in analyses that use aggregate (study-level) information, due to the risk of ecological bias that occurs when the association at the group level does not reflect the underlying association at the individual level.^{8 9} Individual data, either at the level of a trial¹⁰ or of a meta-analysis,¹¹ are necessary to examine effect modification by individual characteristics.

The recent upsurge of research in internet-delivered CBT (iCBT) provides a unique opportunity to systematically review and quantitatively disentangle the efficacy of the various components of CBT. First, in iCBT, constituent components of CBT are easily identifiable, and it is guaranteed that they were made available to the participants, unlike in face-to-face CBT. Second, new methods to analyse complex interventions and synthesise their findings^{12 13} can then be applied to randomised controlled trials (RCTs) in iCBT. In this study, we will apply component network meta-analysis (cNMA), a newly developed meta-analysis method, where various components of different therapies can be dismantled and compared while taking advantage of the whole network of randomised evidence.^{14 15} We will extend existing cNMA models to include individual participant data (IPD) in the meta-analysis.^{16 17} This will allow us to identify and explore the impact of participant-level prognostic factors (PFs; variables that affect the disease progression equally for all the treatments in the network) and effect modifiers (EMs; variables that have an impact on the relative effects of interventions).

This study therefore aims to uniquely overcome the complexity in psychotherapy research at the levels of intervention and population by identifying the more effective components of iCBT and by pinpointing the individual patient’s characteristics that modify their effects. Such findings will help develop more effective and more efficient forms of iCBT by focusing on the best-performing components and facilitate personalised applications of iCBT, aiming to maximise the therapeutic effect while better matching the administered treatment to the individual patients’ characteristics, needs and preferences.

Methods

The protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension statement for network meta-analysis,¹⁸ individual participant data meta-analysis¹⁹ and study protocol.²⁰

Eligibility criteria

We will include all RCTs that compared any form of iCBT against another form of iCBT or a control intervention in the treatment of adults with depression.

We will include adult patients of both sexes aged 18 years or older, with a primary diagnosis of depression, either diagnosed as unipolar major or minor depression according to operationalised diagnostic criteria including Diagnostic and Statistical Manual 3rd Edition (DSM-III), 3rd revised edition (DSM-III-R), 4th edition (DSM-IV), 5th edition (DSM-5), International Classification of Diseases 10th revisioin (ICD-10) or any similar criteria, or judged so by elevated scores on any self-report depression scales. The effect of including studies without formal diagnosis of major depression will be tested in a sensitivity analysis. Inpatients as well as participants with bipolar depression or with psychotic depression will be excluded. Studies including participants with depression comorbid with another mental disorder will be included as long as their study inclusion criteria do not specifically include such comorbidities; however, we will exclude studies focusing on depression comorbid with another mental disorder, because in such cases, the CBT interventions would usually have components that target at such comorbidities in addition to depression. If studies include patients with depression or another mental disorder, we will include them if and only if we can focus on participants with depression based on individual data. Studies focusing on depression with another physical disorder (eg, diabetes and parkinsonism) or in special populations (eg, elderly, pregnancy, mother of autistic children and ethnic minority) will be included; however, the effect of including such studies will be examined in a sensitivity analysis. If a minority of the participants satisfy any of the above exclusion criteria, we will do our best to exclude such participants by employing IPD: when we do not have IPD or cannot exclude them using IPD because details are not available, then we will include such studies in our analysis when they constitute less than 20% of the total population.

iCBT must be a web-based or app-based programme using the internet to deliver the CBT contents. Computerised CBT will be included if it allows interaction between the programme and the individual. Telephone CBT will be excluded. If the use of the internet is limited to teleconferencing/videoconferencing, emails or text messaging, such programmes will also be excluded. When a web-based or app-based programme is used in the context of face-to-face sessions (so-called blended treatment), such programmes will be excluded because then the delivery of the CBT contents is no longer as assured as in pure iCBT. Encouragement to proceed with iCBT by telephone or face-to-face contact limited to an initial evaluation or orientation session will be allowed and considered to be a component of iCBT programme. Studies will be excluded if the iCBT programme allows participants to choose among the available components because such studies will not answer our clinical question.

The control conditions of interest will include the waiting list control, no treatment control, attention/psychological placebo control and treatment as usual. Different studies call different conditions as treatment as usual.²¹ In this cNMA study, treatment as usual must include pharmacotherapy; watchful waiting or follow-up by community nurses will be classified as attention/psychological placebo even when it is ‘treatment as usual’ in some settings. Pill placebo control will not be included in the present network as it is not decomposable into the components of our interest.

This study focuses on the acute phase treatment of the above-defined depression. The intervention can be of any duration; however, the influence of duration will be examined in a meta-regression.

We conceptualise CBT broadly as a psychotherapy involving any one of the following cognitive or behavioural skills’ training. Table 1 presents the different components of interest and their definitions. We will include all intervention or control conditions as long as they could be regarded as a combination of these components. Pharmacological coadministration will be allowed so long as there are no systematic differences in drug administration between study arms, and its presence will be denoted by dt (for definition see table 1). Table 2 provides how various forms of iCBT can be conceptualised from the component perspective.

Table 1.

List of included components and their definitions

w	Waiting component	Participants are aware that they can receive an active treatment after a waiting phase. Usually patients on a wait list do not receive any sort of treatment during the waiting phase. However, in some trials patients allocated to the waiting list control condition receive some non-specific therapeutic components such as psychological placebo, psychoeducation or treatment as usual while waiting. In such cases, we will consider that the ‘waiting component’ (w) is present and also record the interventions provided while waiting. Sometimes publications are not clear if their control conditions allowed the intervention to be administered after the last assessment: in such cases, we need to check the published protocol, trial registries, secondary publications from the same trial and/or ask the original authors.
dt	Conventional drug treatment	Treatment as usual or care as usual can denote many different conditions in the literature.21 In this study, we focus on the presence of ‘conventional drug treatment’ and extract the data if it is present (drug treatment is part of the protocol treatment), allowed (we will note the percentage of patients on drug) or absent.
pl	Placebo effect	Effect of an intervention due to the patients’ belief that they are receiving some form of treatment. In trials, Hawthorne effect can be considered to be always present because all the participants, in the intervention or the controls, will be evaluated. We therefore will drop considering Hawthorne effect as a component in this component NMA. There will be a few trials that may teach skills not covered in this classification (eg, ‘expressive writing’ and ‘dreamwork’). We assume such miscellaneous interventions to have some placebo effect. However, it is possible that some of them may possess some effect beyond placebo effect. We will examine the robustness of our assumption by conducting a sensitivity analysis excluding any studies that taught such miscellaneous skills not categorisable here.
pe	Psychoeducation about depression	Provision of information about the cause and nature of depression. Patients are taught their symptoms can be interpreted under a certain psychopathological model. For example, if cognitive distortion is cited as the cause of depression, such explanation will count towards pe as defined here. Advice about lifestyle modification (eg, exercise, food and sleep hygiene (as opposed to CBT for insomnia)) will be regarded as form of psychoeducation. Provision of information about depression in informational websites will count towards psychoeducation.
cr	Cognitive restructuring	This component teaches the patient to evaluate and modify their own irrational, maladaptive or dysfunctional thoughts using strategies such as Socratic questioning and guided imagery.
ba	Behavioural activation	This component aims at helping people increase potentially reinforcing experiences through activity scheduling and increased engagement in pleasant activities.
is	Interpersonal skill training	Training in appropriate social behaviours. Includes assertiveness training that teaches the patient to stand up to their own rights by expressing their feelings and wishes in an honest and respectful manner that does not insult or hurt the others.
ps	Problem solving	This skill includes the following step-by-step approach to personal problems: defining personal problems, generating multiple solutions, selecting the best solution, working out a systematic plan for this solution and evaluating whether the solution has resolved the problem.
re	Relaxation	This skill is aimed at reducing general tension through induction of a relaxed body state. The most common technique is Jacobson’s progressive muscle relaxation or applied relaxation.
3w	Third-wave components	Various techniques aimed at helping patients to develop more adaptive emotional responses to situations, such as the ability to observe symptomatic processes without overly identifying with them or without reacting to them in ways that cause further distress.56 Some typical examples include training in mindfulness, self-compassion or acceptance.
bi	Behaviour therapy for insomnia	This skill aims at treating chronic insomnia based on the principles of sleep restriction and stimulus control. It may also involve cognitive restructuring around maladaptive beliefs for sleep. It may also involve teaching sleep hygiene; however, sleep hygiene only would count towards lifestyle modification.
rp	Relapse prevention	Review of learnt skills and listing action plans for the foreseeable future problems based on the skills learnt. A mere explanation of relapse in depression will only count towards psychoeducation; in order to qualify for relapse prevention component, it needs more participation from the patient.
hw	Homework required	When completion of some homework assignment is required (or explicitly encouraged repeatedly) before proceeding with the programme, either checked by humans or mandated by the programme. The homework must pertain to exercise in applying the learnt CBT or other skills in one’s own situations and must require some active participation from the participant. Simple reviewing of the materials or further reading will not be regarded as homework.
ff	Initial face-to-face contact	Initial face-to-face human contact, such as the initial evaluation session or the initial orientation session, is present. In conventional drug treatment, ff is considered to be present.
ae	Automated encouragement to proceed with iCBT	Provision of automated, fixed prompts/encouragements to proceed with the treatment programme. Such prompts should not contain any support related to the therapeutic contents.
he	Human encouragement to proceed with iCBT	Prompts/encouragements are prepared and provided by human beings to proceed with the treatment programme via telephone or email. Such prompts should not contain any support related to the therapeutic contents. Peer support such as discussion group will be counted towards this component.
tg	Therapeutic guidance for iCBT	Guidance as to the contents of iCBT. Therapeutic guidance related to the treatment content may be provided on a scheduled basis or as-needed basis. Provision of technical support only is not counted towards this component.

iCBT, internet-delivered cognitive–behavioural therapy; NMA, network meta-analysis.

Table 2.

Conceptualisation of some representative forms of iCBT or control conditions according to a component-level perspective

Interventions or controls	Possible decompositions into components
Waiting list	w (±pl±pe±dt)±ff
No treatment	±ff
Attention/psychological placebo (APP)	pl±ff
Treatment as usual (TAU)*	pl+dt±ff
Psychoeducation (PE)†	pl+pe±rp±dt±(ae/he)±tg±ff
Relaxation therapy†	pl(±pe)+re±rp±dt±(ae/he)±tg±ff±hw
Cognitive therapy†	pl (±pe±re)+cr±rp±dt±(ae/he)±tg±ff±hw
Behavioural activation (BA)†	pl (±pe±re)+ba±rp±dt±(ae/he)±tg±ff±hw
Problem-solving therapy†	pl (±pe±re)+ps±rp±dt± (ae/he)±tg±ff±hw
Assertiveness training†	pl (±pe±re)+at±rp±dt±(ae/he)±tg±ff±hw
Cognitive-behavioural therapy (CBT)†	pl (±pe±re)+cr+(ba±ps±at±bi)±rp±dt±(ae/he)±tg± ff±hw
Third-wave CBT (3W)†	pl (±pe±re±cr±ba±ps±at±bi)+3w±rp±dt±(ae/he)±tg±ff±hw

*TAU here must include pharmacotherapy. Watchful waiting or follow-up by community nurses will therefore be classified as APP even when it is ‘treatment as usual’ in some settings.

†Any of these active interventions may be provided with or without TAU.

3W, third-wave components; ae, automated encouragement to proceed with iCBT; bi, behaviour therapy for insomnia; cr, cognitive restructuring; dt, conventional drug treatment; ff, initial face-to-face contact; he, human encouragement to proceed with iCBT; hw, homework required; iCBT, internet-delivered cognitive–behavioural therapy; is, interpersonal skill training; pe, psychoeducation about depression; pl, placebo effect; ps, problem solving; re, relaxation; rp, relapse prevention; tg, therapeutic guidance for iCBT; w, waiting component.

Study identification and selection

We will use an existing database of psychological treatments for depression that is updated annually through comprehensive literature searches in the bibliographic databases of PubMed, PsycINFO, Embase and the Cochrane Library.²² The search strings use a combination of index and free terms of psychological treatments and depression.

Two independent researchers will check this database for relevant studies according to the eligibility criteria. Any disagreement will be resolved by discussion and, where necessary, in consultation with a third reviewer.

A further literature search will be conducted for studies published since the last update of the database in PubMed and the Cochrane Library. In addition, we will also check the primary studies from recent meta-analyses of internet treatments for depression to ensure that no published studies will be missed. We will also ask the primary authors of the eligible studies if they are aware of any other study that has been conducted in the examined field.

Data collection and integrity checks

Authors of the eligible studies will be contacted and requested to contribute their individual-level data. The corresponding author will be contacted first; if unreachable, a follow-up email will be sent to the senior author of the study. Reminders will be sent after 2 weeks and if necessary after 4 weeks. If no response is received after additional 4 weeks, this trial will be classified as ‘IPD unavailable’ and will be included in the analyses at the aggregate data level.

Authors will be asked to provide the individual-level raw data for their primary depression measures at baseline and at end of the acute phase treatment defined by the original study authors, as well as other potentially important covariates (PFs and EMs of treatment outcome; see section below).

After collecting all primary individual-level data, two independent reviewers will cross-examine the obtained data against the summary statistics (numbers and percentages or means and SDs) of the baseline demographic and clinical variables as reported in the publications of each study. In case the numbers do not match, we will contact the authors of the trials for clarification.

Identification of components

Two independent reviewers will determine the classification of all identified arms and their constituent components according to the definitions in tables 1 and 2, based on all available information including the publications, the trialled iCBT programmes and inquiry with the original investigators. Any disagreement will be solved by the two reviewers and, where necessary, in consultation with a third member of the review team.

Outcome measures

Our primary outcome is:

1. Depression severity as measured on a continuous scale for depression at the end of the acute phase treatment. We will include change in scores from pretreatment to post-treatment on any validated depression outcome measure, such as the Beck Depression Inventory, Beck Depression Inventory-II, Center of Epidemiological Studies Depression Scale, Patient Health Questionnaire-9 or Hamilton Rating Scale for Depression. If a study uses more than one depression measure, preference will be given to the measure reported by the majority of the included studies. In case a study reports two or more outcome measures, none of which are used by the rest of the included studies, preference will be given to the measure listed as primary in this study. If the studies use different outcome measures, they will be converted into the most commonly used scale using the established conversion algorithms.^{23 24} If this approach cannot cover a substantial proportion of the obtained data, scale scores will be standardised (transformed into z-scores) to create a common metric for depression severity.^25–27

We will also examine the following two secondary outcome measures.

1. Dropouts from the end-of treatment assessment for any reason, as a proxy measure of treatment acceptability.

2. Dropouts from the treatment, defined as completing less than 80% of the contents of the programme. If the original authors used a different threshold/definition for ‘completion’ of the programme, we will use their definition.

PFs and EMs of treatment outcome

In this study, we will start from a wide range of patient-level variables and explore their role as either PFs or EMs. We will initially select candidate covariates based on previous literature findings and the availability of these variables in the included studies. The following is the list of candidate variables based on the published literature.²⁸

Demographics

1. Sex.

2. Age.²⁹

Life and social history

1. Childhood maltreatment.³⁰

2. Education.³¹

3. Employment.^{2 32}

4. Marital status.^{2 32 33}

5. Recent life events and difficulties.^{2 32}

6. Social adjustment/function.³⁴

History of present illness

1. Age at onset.³⁵

2. Duration of current episode.²⁹

3. No of previous episodes.^{31 36}

4. Prior treatment with antidepressants.²

5. Prior treatment with psychotherapies.

Present illness: symptomatology

1. Baseline severity.^37–39

2. Baseline psychomotor symptoms.^{34 40}

3. Baseline anxiety symptoms.^{40 41}

4. Baseline somatic anxiety.³⁴

5. Comorbid personality disorder.²

6. Comorbid alcohol or substance use/abuse.⁴⁰

The following study-level characteristics will also be examined as PF or EM.

Study characteristics

1. Duration of intervention.⁴²

2. Presence of inactive control condition.⁴³

It can be expected that different studies use different scales or different categorisations to measure the same or similar constructs. Some measures (eg, social adjustment and baseline anxiety) may be standardised to arrive at a common metric; others may need be dichotomised (eg, employment status, marital status) to harmonise the covariates in the analyses.

Risk of bias assessment in individual studies

Two independent raters will assess the risk of bias in the included studies using the tool described in the Cochrane Collaboration Handbook⁴⁴ as being at high risk of bias, low risk of bias or unclear risk of bias in the following domains: generation of allocation sequence, allocation concealment, blinding of study personnel and participants, blinding of outcome assessor, attrition, selective outcome reporting and other domains. Disagreements between the two independent assessors will be solved through discussion; when there still remains doubt, we will go back to the original authors for clarification.

Note that blinding of study personnel and participants is usually impossible in psychotherapy research; however, in studies of iCBT, it may be at least theoretically feasible if alternative active treatments are provided simply as ‘active treatments’. Note also that attrition bias will be evaluated in the case of studies with individual-level data, not according to the published report, but according to the provided dataset after missing data are imputed as per the statistical methods below. Some studies may still be rated at high risk of bias after imputation if large and unbalanced dropouts exist in the provided raw data. In the case of studies without individual data, attrition bias will be rated according to the publication.

In the above risk of bias assessment, we chose not to evaluate treatment fidelity as it will not be an issue with iCBT, which is fully structured. We also chose not to evaluate allegiance as its measurement is still controverted and will be especially difficult to measure in the comparisons against control conditions⁴⁵ and may be less important in high-quality studies.⁴⁶

Patient and public involvement

There was no patient or public involvement in the development of this manuscript.

Synthesis methods

We will start by performing an aggregated data network meta-analysis (NMA) on the treatment level to gain a first insight of the relative treatment effects.⁴⁷ We will do this analysis both with and without trials for whom we do not have IPD.

Then we will continue to our main analysis. This will be done in two steps. In the first step, we will perform a variable selection procedure to decide which of the candidate covariates to include in the evidence synthesis model of the second step. To this end, we will fit a penalised linear regression model with an elastic net penalty⁴⁸ to our primary outcome (continuous efficacy). We will explore all candidate covariates as well as their interactions with the components, aiming to identify the most important patient characteristics in terms of both prognosis and effect modification. Aiming to facilitate convergence of the models and to help the interpretation of the results of the second step, all continuous covariates will be standardised, that is, by subtracting the mean and dividing by the corresponding SD. The output of this first step of the analysis will be a list of PFs and EMs. Note that different EMs may be associated with different components, for example, patient age may be found to strongly interact with cognitive restructuring but not with behavioural activation.

At the second step, we will fit a one-stage IPD-NMA model.^{16 17} Let us assume that study ${\displaystyle j}$ compares two interventions ${\displaystyle T_1}$ and ${\displaystyle T_2}$, where ${\displaystyle T_1}$ comprises components ${\displaystyle c_1}$ and ${\displaystyle c_2}$ and ${\displaystyle T_2}$ comprises components ${\displaystyle c_3}$ and${\displaystyle c_4}$. Let us assume that patient ${\displaystyle i}$ in this study had an observed outcome ${\displaystyle y_{ij}}$. Let us also assume that for this patient we have the corresponding PFs in the form of a vector ${\displaystyle \mathit{P}{\mathit{F}}_{\mathit{i}\mathit{j}}}$. Similarly, let us assume that the EMs for this patient, for each component, are in vectors ${\displaystyle \mathit{E}{\mathit{M}}_{\mathit{i}\mathit{j}}^{\left({\mathit{c}}_1\right)}}$, ${\displaystyle \mathit{E}{\mathit{M}}_{\mathit{i}\mathit{j}}^{\left({\mathit{c}}_2\right)}}$, ${\displaystyle \mathit{E}{\mathit{M}}_{\mathit{i}\mathit{j}}^{\left({\mathit{c}}_3\right)}}$ and ${\displaystyle \mathit{E}{\mathit{M}}_{\mathit{i}\mathit{j}}^{\left({\mathit{c}}_4\right)}}$. The model can now be written as follows:

$${\displaystyle y_{ij}\sim N\left(z_{ij},s_j^2\right)}$$

$${\displaystyle z_{ij}=\{\begin{array}{l}{u}_j+\beta \mathit{P}{\mathit{F}}_{\mathit{i}\mathit{j}}+{\mathit{\gamma }}^{\left({\mathit{c}}_1\right)}\mathit{E}{\mathit{M}}_{\mathit{i}\mathit{j}}^{\left({\mathit{c}}_1\right)}+{\mathit{\gamma }}^{\left({\mathit{c}}_2\right)}\mathit{E}{\mathit{M}}_{\mathit{i}\mathit{j}}^{\left({\mathit{c}}_2\right)},iftrea{t}_{ij}=T_1=\left(c_1+c_2\right)\\ u_j+\beta \mathit{P}{\mathit{F}}_{\mathit{i}\mathit{j}}+{\mathit{\gamma }}^{\left({\mathit{c}}_3\right)}\mathit{E}{\mathit{M}}_{\mathit{i}\mathit{j}}^{\left({\mathit{c}}_3\right)}+{\mathit{\gamma }}^{\left({\mathit{c}}_4\right)}\mathit{E}{\mathit{M}}_{\mathit{i}\mathit{j}}^{\left({\mathit{c}}_4\right)}+{\delta }_j,iftrea{t}_{ij}=T_2=\left(c_3+c_4\right)\end{array}}$$

In this expression, ${\displaystyle \mathit{\beta }}$ is the vector of regression coefficients associated to PFs. ${\displaystyle s_j^2}$ is the study-specific variance of ${\displaystyle y_{ij}}$. ${\displaystyle {\mathit{\gamma }}^{\left({\mathit{c}}_{\mathit{X}}\right)}}$ is the vector of regression coefficients for effect modification (component–covariate interaction), for component${\displaystyle X}$. The ‘baseline’ effect, ${\displaystyle u_j}$, will be assumed exchangeable across studies, that is, ${\displaystyle u_j\sim N\left(m_u,{\sigma }_u^2\right).{\delta }_j}$ is the study-specific estimate of relative effect for ${\displaystyle T_2}$ versus ${\displaystyle T_1}$, at the zero values of the (possibly standardised) EMs, that is, when all ${\displaystyle EM=0}$. We will assume that ${\displaystyle {\delta }_j\sim N\left({\mu }_j,{\tau }^2\right)}$, where ${\displaystyle \tau }$ denotes the random effects SD in the network. We will assume ${\displaystyle \tau }$ to be common for all treatment comparisons in the network. Finally, ${\displaystyle {\mu }_j}$ will be expressed as a linear combination of the effects of the components, for example, for this particular example we will set ${\displaystyle {\mu }_j=d_3+d_4-d_2-d_1}$. This model assumes additivity of treatment effects (Welton et al ¹⁴). In order to include in our analysis patients with missing values for one or more of the covariates (both PFs and EMs), we will use a study-specific imputation scheme. For example, if for patient ${\displaystyle i}$ in study ${\displaystyle j}$ there is no information regarding age, we will stochastically impute it by drawing from a study-specific distribution, that is, ${\displaystyle ag{e}_{ij}\sim N\left(\stackrel{-}{ag{e}_j},s_{age,j}^2\right)}$. Here ${\displaystyle \stackrel{-}{ag{e}_j}}$ denotes the mean age of patients in this study and ${\displaystyle s_{age,j}^2}$ the corresponding variance.

For the secondary outcome ‘dropout from end-of-treatment assessment’, we will use a binomial likelihood for the observed data. We will synthesise data on the OR scale. The rest of the model will be as for the primary outcome.

For the secondary outcome ‘dropout from the treatment’, we will use a different modelling approach. This is because, by definition, for the inactive control conditions, treatment dropout is not observed. This implies that in studies with inactive controls, we cannot infer about relative effects. Such studies are expected to represent the vast majority of all eligible studies. Thus, for the analysis of this outcome, we will synthesise absolute treatment effects. We will model the probability of dropout for each active treatment, using a generalised linear model with a binomial likelihood

We will implement the variable-selection procedure of our analysis (step one) in R,⁴⁹ using the glmnet package.⁵⁰ The variable-selection procedure will only be performed for the primary outcome. We will implement the evidence-synthesis part (step 2) in OpenBUGS.⁵¹

Limitations of the proposed statistical model

The basic model assumes additivity of components and does not take account of possible interactions among components, that is, when some components may be particularly effective or ineffective in combination with some other components. The dataset will likely lack statistical power to test for such interactions. However, for components that are well represented in the network (ie, those which have been studied in many trials involving many participants), we will run exploratory analyses of some representative interactions.

The proposed model cannot discern the effect of the ordering of the components, but we expect that we will not have enough relevant data from the studies to explore this effect. If enough data become available, we will modify our model to explore the ordering effect among the most well-represented components of the network.

Heterogeneity and inconsistency

We will measure heterogeneity in the included studies by estimating a common heterogeneity parameter ${\displaystyle \tau }$, as described in the previous section. We will compare this estimate with its empirical distribution,^{52 53} for the dropout outcome. For the continuous outcome, we will compare ${\displaystyle \tau }$ with an empirical distribution (Rhodes et al ⁵²) only if we use standardised scores (see section Outcome measures).

We expect that the inclusion of the covariates in the model will lower the observed inconsistency, that is, by explaining some of the observed differences between the studies. In order to assess this, we will also fit a model without patient covariates and report any changes in ${\displaystyle \tau }$.

We will assess inconsistency in the network at the treatment level by estimating the difference between direct and indirect evidence. For this, we will use the design-by-treatment inconsistency model (Higgins et al ⁴⁴). We will then also check inconsistency at the component level and report on any important differences.

Data availability bias

As discussed above, we anticipate that for at least some of the identified trials, we might not be able to obtain IPD. If studies providing IPD are systematically different from studies not providing IPD, there may be doubts regarding the validity of our findings. We will formally assess the agreement between these two sets of trials by analysing them separately and subsequently comparing the corresponding results. We will report any important discrepancies, and we will take them into consideration when we evaluate the quality of the evidence provided by our analysis.

Publication bias

We will first examine small study effects by visually inspecting the contour-enhanced funnel plots of pairwise meta-analyses for efficacy when 10 or more studies per comparison are available. We will test for small study effects using Egger’s test for the continuous outcome.⁵⁴

Sensitivity analyses

The following five sensitivity analyses will be conducted for the primary outcome.

1. We will examine the impact of studies without formal diagnosis of depression by excluding such studies from the analyses.

2. We will examine the impact of studies focusing on patients with depression and a physical disorder by excluding such studies from the analyses.

3. We will examine the impact of miscellaneous skills not covered under any category in our classification in table 1 by excluding studies that included such skills.

4. We will run a sensitivity analysis by limiting to studies where at least 60% of the participants have completed at least 80% of the programme in order to exclude the influence of trials where the completion rate may have been particularly low due to some external circumstances that are not inherent to the components themselves.

5. Our main analyses regarding depression severity will only use information from patients for whom the outcome was reported. This corresponds to a ‘missing completely at random (MCAR)’ assumption.⁵⁵ In this sensitivity analysis, we will explore a ‘missing not at random’ scenario, where we will assume that the probability of dropping out from assessment is affected by the (unobserved) depression severity. More specifically, we will use a selection model described by Debray et al,¹⁶ where patients with worse outcomes are more likely to drop out. We will perform this sensitivity analysis only if the dropout rates are high (>50%).