Individual patient data meta-analysis of combined treatments versus psychotherapy (with or without pill placebo), pharmacotherapy or pill placebo for adult depression: a protocol

Abstract

Introduction

There are many proven treatments (psychotherapy, pharmacotherapy or their combination) for the treatment of depression. Although there is growing evidence for the effectiveness of combination treatment (psychotherapy + pharmacotherapy) over pharmacotherapy alone, psychotherapy alone or psychotherapy plus pill placebo, for depression, little is known about which specific groups of patients may respond best to combined treatment versus monotherapy. Conventional meta-analyses techniques have limitations when tasked with examining whether specific individual characteristics moderate the effect of treatment on depression. Therefore, this protocol outlines an individual patient data (IPD) meta-analysis to explore which patients, with which clinical characteristics, have better outcomes in combined treatment compared with psychotherapy (alone or with pill placebo), pharmacotherapy and pill placebo.

Methods and Analysis

Study searches are completed using an established database of randomised controlled trials (RCTs) on the psychological treatment of adult depression that has previously been reported. Searches were conducted in PubMed, PsycInfo, Embase and the Cochrane Central Register of Controlled Trials. RCTs comparing combination treatment (psychotherapy + pharmacotherapy) with psychotherapy (with or without pill placebo), pharmacotherapy or pill placebo for the treatment of adult depression will be included. Study authors of eligible trials will be contacted and asked to contribute IPD. Conventional meta-analysis techniques will be used to examine differences between studies that have contributed data and those that did not. Then, IPD will be harmonised and analysis using multilevel regression will be conducted to examine effect moderators of treatment outcomes.

Dissemination

Study results outlined above will be published in peer-reviewed journals. Study results will contribute to better understanding whether certain patients respond best to combined treatment or other depression treatments and provide new information on moderators of treatment outcome that can be used by patients, clinicians and researchers.

Trial registration number

CRD42016039028.

Strengths and limitations of this study.

• This is the first individual patient data (IPD) meta-analysis of combined treatment for depression versus pharmacotherapy alone, psychotherapy alone or psychotherapy plus pill placebo for depression.

• Using IPD meta-analysis methods will allow for examination of individual patient's clinical and demographic characteristics as moderators between combined treatment and comparator treatments for depression by maximising statistical power while protecting against ecological fallacies that present problems when examining aggregate data using conventional meta-analysis techniques.

• This study can contribute important information towards identifying factors that affect response to varying depression treatments.

• However, the IPDMA is limited to only examining factors that are reported similarly across all of the included individual studies.

Introduction

There are many evidence-based treatments for depression such as various psychotherapies like cognitive behaviour therapy (CBT), behavioural activation (BA), interpersonal therapy (IPT), problem-solving therapy (PST) and psychodynamic therapy^1–5 and there are various classes of antidepressant medications such as the selective serotonin reuptake inhibitors (SSRIs), serotonin-norepinephrine reuptake inhibitors (SNRIs), tricyclic antidepressants (TCAs) and monoamine oxidase inhibitors (MAOIs).⁶ Many of these treatments have been found to be as effective as monotherapy and to be comparable with one another.⁷ Researchers and treatment guidelines generally agree that either type of monotherapy may be useful in the treatment of mild to moderate depression; however, treatment guidelines suggest a combination of psychotherapy and pharmacotherapy for the treatment of more moderate to severe depression.^8–10 In addition, there is growing evidence from randomised controlled trials (RCTs) and conventional meta-analyses that combination treatment is more effective for the acute phase treatment of depression than psychotherapy alone,^{11 12} pharmacotherapy alone¹³ and psychotherapy plus pill placebo.¹⁴

Although much is known about how well depression treatments work on average, less is known about how these treatments work at the level of the individual patient. For instance, different treatments may be comparably effective for the average patient, yet some patients may improve more on a combination of treatments than a certain monotherapy.¹⁵ Factors that can predict differential response between two treatments are known as effect modifiers or moderators.¹⁶ Similarly, many patients respond as well on a specific monotherapy as they do in combined treatment, therefore, using combined treatment for these patients would waste valuable economic resources given that combined treatments are much more costly to provide. Knowing under which circumstances an individual with a certain characteristic would have a superior response to a monotherapy or to combination treatment would have important implications for clinical practice and subsequent research (specific response points to specific causal mechanisms), and would add to the growing body of evidence moving towards what is frequently called personalised medicine.^17–19

In order to determine which patients respond best to which treatments, it is necessary to examine individual baseline clinical (depression severity, psychopathological comorbidities, depression chronicity, previous exposure to treatment, etc) and demographic characteristics more closely. Few RCTs have examined these individual characteristics as moderators of differential response in depression treatment outcomes between combined treatment versus monotherapy, control conditions or psychotherapy plus pill placebo. In those trials that have examined individual characteristics as moderators of differential response between combination and comparison treatments, baseline depression severity was the most commonly examined variable with mixed results. One recent trial found that combined therapy was worse than pharmacotherapy alone for those with severe depression,²⁰ whereas another found that patients with severe depression had an increased rate of recovery in combination treatment versus pharmacotherapy alone.²¹ In addition, a meta-analysis that examined studies including less severe patients and studies incorporating more severe patients found greater remission rates in combined treatment compared with psychotherapy alone for those with severe or chronic depression, but no difference in those with mild depression.¹² Demographic and other clinical variables have gone largely unassessed as moderators between combined treatment and psychotherapy, pharmacotherapy or pill placebo monotherapy or psychotherapy plus pill placebo combination treatment in RCTs, but have been examined more thoroughly in RCTs of psychotherapy and pharmacotherapy with some success.^{15 22–26} However, the problem with this method is that these trials often have smaller sample sizes and limited statistical power to detect significance of these variables without encountering a type I or type II error. To the best of our knowledge, no studies of combination treatment versus pill placebo alone or psychotherapy plus pill placebo have documented any moderating variables.

Conventional meta-analysis techniques are commonly used to aggregate outcome data of RCTs of depression. However, these techniques often cannot be used appropriately when examining moderation since data may not be reported in published papers, or may be reported differently across trials, which prevents aggregation. When aggregation is possible in meta-analysis, it is often by use of subgroup analysis and this can also limit statistical power and accuracy of analysis since it leads to a loss of degrees of freedom and variability in the moderator of interest that may lead to ecological fallacies. Therefore, conducting an individual patient data (IPD)meta-analysis, by collecting and aggregating the raw IPD from RCTs is necessary in order to better understand the predictive nature of individual characteristics on the difference between combination treatment and psychotherapy or pharmacotherapy monotherapy, pill placebo or psychotherapy plus pill placebo for the treatment of depression.

IPD meta-analysis techniques have been used with some frequency in medicine,²⁷ but are newer in the field of clinical psychology and psychiatry. IPD methods can offer several advantages in summarising existing evidence from individual trials. As the field moves towards personalised medicine, being able to select the best treatment for groups of patients with certain characteristics, IPD methods can be a useful tool for examining moderators of varying outcomes with sufficient power. Although it is possible that IPD meta-analysis will not uncover significant moderators of interest even with additional power, this would be an equally important finding for the field of personalised medicine.

IPD meta-analyses also present many challenges. These methods are more time and resource intensive than conventional meta-analyses. Using these methods is dependent on accessing raw data from researchers and it involves making complex decisions on data to ensure accuracy of outcomes. IPD methods are described in detail in this protocol, which outlines the design of an IPD meta-analysis of combined treatment compared with psychotherapy (with or without pill placebo), pharmacotherapy or pill placebo for adult depression. The main objective of this meta-analysis is to determine which patients respond better to combined treatment (psychotherapy + pharmacotherapy) compared with monotherapies (pharmacotherapy, psychotherapy or pill placebo monotherapy or psychotherapy vs pill placebo combination treatment).

Methods

General study approach

This IPD meta-analysis involves selecting eligible research, collecting relevant data and subsequently, using two separate meta-analytic approaches for data analysis. First a systematic review to identify eligible papers will be performed, studies will be selected and study authors will be invited to contribute data. A conventional meta-analysis will then be performed to test for differences between studies included in the IPD meta-analysis and those that could not provide data. Individual data will be aggregated and previously selected moderator variables will be analysed using a multilevel model approach.

Systematic review to identify eligible papers

Eligibility criteria

Types of studies

This study will include published RCTs. Non-randomised studies will not be included.

Type of participants

Participants of all genders and ethnicities who are 18 years of age or older and who have been diagnosed with a depressive disorder established by a standardised diagnostic interview will be included in this systematic review and meta-analysis. Studies that include populations with comorbid general medical disorders or other psychiatric disorders are not excluded as long as these comorbid disorders are not the primary focus of the study.

Types of interventions

RCTs comparing combination treatment (psychotherapy + pharmacotherapy) with psychotherapy (with or without pill placebo), pharmacotherapy or pill placebo for the treatment of adult depression will be included. Psychotherapeutic interventions are required to be a manualised form of psychotherapy in which there is verbal communication between a therapist and a patient, or where a psychological treatment was written in a systematic format for a patient to follow (in a book or on the internet) with some support from a therapist.²⁸ These will include the major forms of psychotherapy such as CBT, IPT, PST, BA, psychodynamic psychotherapy and others. Pharmacotherapies will include antidepressant treatment such as the SSRIs, SNRIs, TCAs and MAOIs, among others.

Comparison treatments

Eligible comparison treatments will be (1) a psychotherapy as indicated above, (2) an antidepressant pharmacotherapy as indicated above, (3) pill placebo or (4) a combination of psychotherapy and pill placebo.

Types of outcome measures

Treatment efficacy will be measured by standardised depression outcome measures such as the Beck Depression Inventory (BDI);²⁹ Hamilton Depression Rating Scale (HAM-D);³⁰ Montgomery-Asberg Depression Rating Scale (MADRS)³¹ or other validated depression measures. Preference will be given to measures listed as primary outcome measures in the protocols. If two primary outcome measures are used, preference will be given to blinded assessments (clinician-interviewed over self-report measures). If the type of outcome measures used varies between studies, these measures will be transformed into standardised z-scores to retain their properties as continuous measures and also dichotomised to reflect common clinical criteria such as response (a 50% reduction in symptoms at post-treatment) and remission (maximum absolute scores reflecting normalisation). They also will be dichotomised to reflect extreme response and non-response/deterioration.³²

Types of predictor/moderator variables

Published papers will be examined to determine valid predictors reported across studies. This project will focus on clinical and demographic moderators of treatment outcomes including correlates of depression severity. Treatment guidelines recommend combined treatment for patients with severe depression, thus suggesting that there is a differential effect of treatment (combined vs monotherapy) as a function of depression severity. In addition, each of the particular moderator variables selected has been examined in previous RCTs and has been found to predict or moderate treatment outcomes in depression. The clinical predictors that will be examined in this study are: baseline depression severity^{25 33 34} measured on the measures outlined above, having a comorbid mental health diagnosis,^{15 23} anxiety symptoms,²³ number of previous episodes (recurrence)^{26 35} length of current episode (chronicity),¹⁷ global assessment of functioning (GAF)²⁴ and previous exposure to depression treatments.¹⁵ Demographic moderators that will be examined in this study are: marital status,^{15 22 36} employment,^{15 22} education²⁶ and age.¹⁷ Other baseline demographic characteristics will be gathered in order to adjust the analysis for these baseline characteristics. In addition, previous literature has found that social adjustment²⁴ predicted outcomes, and thus will be included when available. It is expected that not all studies assessed will be able to contribute all variables, and thus, indices will be selected when they uniquely examine a clinical correlate of interest (ie, are not similar to another variable included) and when a majority of studies have provided this particular data.

Timing of outcome assessments

All acute phase (postintervention) outcomes (between 5 and 36 weeks) will be included despite potential variability in timeframes. If timing of interventions varies extremely, sensitivity analyses examining the effect of length of treatment on outcomes will be conducted. In addition, length of treatment will be included as a control variable in regression analyses. Long-term post-treatment follow-ups will also be included if available and separate analysis will be conducted on the acute phase versus extended follow-ups.

Search methods for identification of studies

Study searches will be completed using an established database of RCTs on the psychological treatment of adult depression. This database has been described previously²⁸ and used in a series of earlier published meta-analyses.³⁷ Comprehensive literature searches were conducted (from 1966 to January 2015) to develop the database that is updated every year in January. These searches identified 16 365 abstracts to be examined from Pubmed, Psychinfo, Embase and the Cochrane Central Register of Controlled Trials. Abstracts and articles that were pulled examined psychological treatments for depression in general. In addition, the authors searched previous meta-analyses of treatments for depression to be sure that no randomised trials were missed in the selection of papers. From the 16 407 abstracts (12 196 after the removal of duplicates), we retrieved 1885 full-text papers of RCTs on treatments for depression for possible inclusion in the database. Thus far, RCTs for depression have been included in the database. These papers were then screened for inclusion in this meta-analysis.

Quality assessment

Study quality will be assessed by using six criteria from the Cochrane Collaboration's ‘risk of bias’ tool.³⁸ Possible risks of bias assessed by this tool include adequate generation of randomisation sequence, concealment of treatment allocation, blinding of assessors, use of appropriate methods for addressing missing data (this was denoted as positive when analysis was completed on the intention-to-treat sample, meaning that all randomised patients were included in the analysis), selective outcome reporting and other sources of bias. Only data from the published papers will be used to determine the risk of bias so as to be consistent across all studies that share data and those who cannot share data. Two independent researchers conduct this quality assessment.

Collecting and aggregating IPD

Inviting authors

All first authors of the identified included studies will be contacted via email with a letter of invitation outlining the project goals and asking if they would be willing to collaborate by sharing the specific raw data from their eligible trial. If an author does not respond after 1 month, a second attempt to contact them via email or post will be made. If the second contact fails, another author of the study will be contacted and invited to participate. A second attempt to contact this author will follow in another month if no response is received and so forth until a maximum of three authors are contacted. Study data will be considered unavailable in the event that no study authors have responded to multiple contact attempts or if all contacted study authors indicate that they no longer have access to the data.

Initial data check

The initial data check will be used to ensure that data received is from the correct trial and is in satisfactory condition to be included in the meta-analysis. Data received will be examined to see that it matches data reported in the published papers. The descriptive statistics from the paper including sample sizes, frequencies of demographic variables, clinical diagnoses and means of depression or anxiety symptom scales will be calculated and compared with the published papers wherever possible. Clarification from authors will be sought when discrepancies arise. If clarification is not available, and the differences are deemed small and judged by three researchers to not have implications for the overall results of the study, the study will be included in the IPD meta-analysis and a sensitivity analysis removing this study will be conducted to ensure inclusion of this study does not affect the results. Studies will be included when they share the necessary data, missing data are not excessive (relative to what is reported in the paper) and there is consensus that study data are accurate.

Database creation

The database will be created in SPSS. Coding for the database will be finalised when all data have been received from the study authors. When a study has coding that differs greatly from the other studies, two researchers will arrive at a group consensus on the recoding and clarification from the study authors will be sought when necessary. A third member of the research team will be consulted when discrepancies arise.

Aggregation

After the initial data checks have been completed, a copy of each trial's raw data will be recoded into a separate database that corresponds with the IPD meta-analysis variables and will be recoded to match the coding of the IPD database. SPSS will then be used to aggregate the individual databases into one large IPD database, structured by study and individual patient ID. After the data have been concatenated, the large IPD database of all studies will again be checked for accuracy.

Analysis

Conventional meta-analysis

A conventional meta-analysis, using data from the published papers, will be conducted in order to compare the outcomes of studies that have contributed data to the IPD meta-analysis with those studies that did not contribute data. The goal of this analysis is to establish if there are any significant differences in depression outcomes, risk of bias and other study characteristics that might bias the IPD meta-analysis. Effect sizes indicating the differences between combination therapy and comparison treatments at post-treatment will be calculated from data reported in the published paper by subtracting the average post-treatment depression score of the comparison treatment group from the combination treatment group and dividing by the pooled SD. If studies use dichotomous outcomes without reporting means and SDs of the continuous depression scores, the effect size calculations for dichotomous variables outlined by Borenstein et al³⁹ will be used. These effect sizes will then be compared in the Comprehensive Meta-Analysis (CMA) software (V.3.0) using the random-effects model because some heterogeneity between studies is to be expected.

In addition, CMA software will be used to perform a standard χ² test to examine the amount of variation across studies that is due to heterogeneity. The I², which expresses the amount of heterogeneity in percentages, will be analysed and low (25%), medium (50%) or high (75%) levels of heterogeneity will be reported.⁴⁰ The 95% CI around I² will be calculated using the heterogi module in STATA. If high heterogeneity is found in the point estimate or CI, further subgroup and metaregression analyses will be provided to explore possible causes of heterogeneity. Small sample bias will be assessed by visually examining the funnel plot and by using Duval and Tweedie's trim and fill procedure which provides an estimated effect size after taking into account bias related to including studies with small samples.⁴¹

Metaregression analyses will be run in CMA in order to examine differences in outcome between studies that contributed data and those that did not. The standardised effect sizes will be the dependent variable and a variable indicating whether data has or has not been shared by the authors, and other study characteristics such as bias score, type of recruitment and other characteristics of the interventions will be entered as the independent variables.

IPD meta-analysis

Primary depression outcome scales and time points in each trial will be selected based on information from the published papers and study authors. When different primary outcome depression measures have been used across the studies, we will convert the depression scores into standardised z scores (by subtracting means from the individual patient score and dividing by the SD within each study and each measure separately) in order to retain continuous scores of depression. However, continuous depression scores will also be converted into response rates per individuals. The universal definition of response is a 50% reduction of scores at post-treatment, thus allowing this outcome to be compared across studies and varying depression outcome measures.

Missing outcome depression scores will be imputed in STATA using valid predictor variables such as individual clinical and demographic characteristics. A one-step IPD meta-analysis approach will be used because it yields less biased estimates and has better performance in terms of power than a two-step approach in which a treatment × moderator interaction is estimated within each trial followed by a standard inverse variance meta-analysis.^{42 43} The aggregated IPD will be examined using multilevel regressions, clustering on the individual study level to take into account any heterogeneity between studies. These multilevel regressions will be used to examine the effects of certain demographic and clinical predictors and moderators on depression outcomes between combination therapy and comparison groups.

In order to examine which patients with what individual characteristics respond better to combined treatment or pharmacotherapy, psychotherapy (with or without pill placebo) or pill placebo, clinical and demographic variables will also be collected. Clinical characteristics such as baseline depression severity as measured by continuous depression measures, comorbid diagnoses, chronicity of depression and demographic variables that are commonly studied such as age, gender, employment and marital status, are of particular interest in this study. Moderator variables will be included in analysis if they are represented in a sufficient number of trials. Moderators found to be significant will be further assessed with subgroup analyses that standardise effect sizes. Effect sizes of standardised mean difference (SMD)=0.24 or above are considered to be clinically relevant.⁴⁴

Sensitivity analysis

Several sensitivity analyses will be included to examine the robustness of the IPD meta-analytic findings. It may be the case that most studies will include an identical or equivalent depression measurement as an outcome variable. If this is the case, then sensitivity analysis using the most used depression measure will be conducted in order to compare analysis in these outcomes with the z-score outcomes. If a sufficient number of trials incorporate HAM-D-17 scores, then a dichotomous variable indicating remission, defined as a HAM-D-17 score of ≤7, will be calculated and analysed as an outcome.

For comparison, similar multilevel models will be employed using only the sample of participants who completed the post-treatment outcome measure. In addition, a third model will be examined that will include individual patient characteristics as control variables.

Sensitivity analysis using individual types of psychotherapy alone will be conducted when there are at least four studies using a particular psychotherapy. This analysis will explore whether moderators are specific to certain types of psychotherapies. The same will be done with respect to placebo combinations. Other sensitivity analyses may be necessary and will be determined after all data have been collected and examined.

Discussion

IPD meta-analysis techniques offer the ability to better aggregate and analyse predictors and moderators of depression outcome among treatments than conventional meta-analysis. Using these models should allow for a better understanding of the effects of patient-level characteristics on depression outcomes than would arise from conventional meta-analysis. Conventional meta-analyses rely on aggregating subgroup analyses reported similarly across all trial RCTs, which rarely occurs. In addition, IPD meta-analysis offers greater statistical power and precision with which to analyse predictors, moderators and differences between outcomes than can individual RCTs, as single trials often are underpowered and thus not able to ascertain statistically significant moderators. This approach also allows researchers to standardise analytical methods across all studies, for instance where some studies may have previously reported only remission rates and others mean depression change over time. These can now be converted from one type of measurement to the other for optimal comparisons.⁴⁵

IPD meta-analysis techniques also present several challenges. First, although they offer significant power to examine moderators of treatment outcome, they must rely on variables previously defined by individual studies. This limits the analysis to exploring moderators that have been collected, are available and are able to be combined across studies. Thus, not all variables of interest can be included. In addition, while recoding variables to be more similar to one another is necessary for the analysis, it is possible that some important details about these variables are omitted from the analysis. Expected barriers to accessing data, such as not finding an optimal method to contact authors or an author's lack of access to data may introduce some bias into the IPD meta-analysis. However, this will be thoroughly examined and addressed by additionally using conventional meta-analysis techniques. Other sources of bias, such as the inability to identify unpublished trials using standard searching methods, may also be present. Unpublished trials in psychotherapy research are difficult to identify without the labour-intensive task of examining records from institutional review boards across all countries, and thus unpublished trial data will likely not be included in this meta-analysis. This may lead to some publication bias and results will need to be interpreted accordingly.

The aforementioned benefits of using IPD meta-analysis techniques greatly outweigh the challenges of using this intensive method. IPD methods allow for a thorough examination of predictors or moderators of treatment outcomes that explain differential treatment effects, in this case between combination treatment and various monotherapies or psychotherapy plus pill placebo for depression. Combination treatment for depression has been proven effective, but it is unclear whether all patients respond to this treatment similarly, or whether some patients benefit more from a certain monotherapy. Previous RCTs have not had sufficient power to thoroughly examine moderators. Thus, although we know depression treatments are equally effective, we do not know whether certain kinds of patients (eg, those who are older or more severe patients) will respond better to a specific type of treatment than another. Knowing which types of patients benefit more from combined treatment than monotherapy can ensure that patients get the optimal treatment and relieve clinicians of the burden to choose the best treatment option for a given patient with very little information to inform that decision. This project aims to contribute this knowledge of which patients respond best to which treatments, to clinicians and researchers in the field of depression treatment.