What works for whom and why? Treatment effects and their moderators among forcibly displaced people receiving psychological and psychosocial interventions: study protocol for an individual patient data meta-analysis

Abstract

Introduction

Forcibly displaced people (FDP) have a high risk of developing mental disorders such as post-traumatic stress (PTS) disorder. Providing adequate mental healthcare for FDP is crucial but despite overall efficacy of many existing interventions, a large proportion of FDP does not benefit from treatment, highlighting the necessity of further investigating factors contributing to individual differences in treatment outcome. Yet, the few studies that have explored moderators of treatment effects are often insufficiently powered. Therefore, the present Individual Patient Data meta-analysis (IPD-MA) will investigate treatment effects and their moderators—variables related to beneficiaries, providers, intervention and study characteristics in relation to PTS outcomes.

Methods and analysis

A systematic literature search will be conducted from database inception in the databases PsycINFO, Cochrane, Embase, PTSDpubs and Web of Science. Only studies published in English, German, French, Spanish, Portuguese, and Dutch will be considered. Retrieved records will be screened for eligibility. Randomised controlled trials on adult FDP receiving psychological and psychosocial interventions aimed at alleviating symptoms such as PTS compared with a control condition without intervention will be included in this IPD-MA. Subsequently, authors of eligible studies will be contacted to request individual patient data (IPD). All datasets obtained will be synthesised into one large dataset which will be analysed using a one-stage approach by conducting mixed-effects linear regression models (ie, primary analysis). Additionally, aggregate data meta-analyes will be run using a two-stage approach by conducting multivariate regression models including all IPD (transformed) and available meta-data from study reports (ie, secondary analysis). PTS will serve as primary outcome measure, while mental health outcomes other than PTS, attendance, attrition, treatment non-response and adverse outcomes will be examined as secondary outcomes.

Ethics and dissemination

This IPD-MA does not require ethical approval. The results will be published in international peer-reviewed journals.

PROSPERO registration number

CRD42022299510.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• This individual data patient meta-analysis (IPD-MA), compared with traditional meta-analyses, will allow more complex analyses to identify moderators of treatment effects at patient level while the standardisation of variables is facilitated and missing values can be accounted for.

• By merging different datasets into one large dataset, this IPD-MA has the potential to investigate predictors of rare events such as adverse outcomes.

• This study can contribute important information towards identifying factors that affect treatment outcome, response, attendance, attrition and adverse effects.

• This IPD-MA is limited by the availability of IPD and their quality.

Introduction

According to the United Nations High Commissioner for Refugees,¹ an unprecedented 108.4 million people worldwide have been forced to flee their homes at the end of 2022 as a result of persecution, conflict, violence and other reasons. Due to a number of ongoing wars and, most recently, the conflict in Sudan, this number has eclipsed 110 million people for the first time.² Forcibly displaced people (FDP) are exposed to many stressors before, during and after displacement.^{3 4} Not surprisingly, FDP are at a high risk of developing mental disorders with estimates, for example, around 32% for post-traumatic stress (PTS) disorder.^{5 6}

Due to the substantial personal suffering and the high economic costs of untreated mental health problems, it is crucial for hosting countries to provide adequate mental healthcare for FDP.⁷ Different treatment approaches have been taken to treat FDP including therapies delivered by specialists (eg, cognitive–behavioural therapy (CBT)⁸), low-intensity interventions delivered by non-specialists (eg, Problem Management Plus (PM+)⁹), and guided (eg, Step-by-Step (SbS)¹⁰) or unguided self-help programmes. The task-sharing approach of scalable psychological interventions delivered by non-specialists seems to be a viable solution for settings which are burdened by a scarcity of specialised mental health services in low-income and middle-income countries¹¹ or where adequate mental healthcare is hindered by language barriers and limited access to facilities in high-income countries.¹² While several meta-analyses have shown different psychological interventions to effectively reduce PTS, there is a considerable heterogeneity among studies,^{13 14} some of which have been investigated and attributed to differences in study characteristics. For example, Gwozdziewycz et al¹⁵ found that treatment effects of narrative exposure therapy increase if the providers themselves have a displacement background. While trials including an active compared to a passive control group seem to be associated with larger treatment effects,^{16 17} findings with regard to treatment dose (ie, number of sessions) tend to be mixed, with evidence for more sessions boosting the treatment effect^{16 17} or having no impact.¹³ However, many tested moderators did not seem to influence treatment effects across studies including medication rate, time since displacement,¹³ residence status,¹⁸ use of interpreter,¹⁶ type of PTS assessment,^{16 17} study quality, country where trial was conducted, or ethnicity.¹⁷

Despite many existing interventions showing overall efficacy, beneficiaries with a forced displacement background compared to those without such a background benefit less from the same interventions,¹⁹ while a large proportion of FDP (up to 60%²⁰) do not improve following treatment. A recent individual patient data meta-analysis (IPD-MA) combining data from several PM+ trials found that although the intervention seemed to effectively reduce PTS among recipients overall, a third of them had persisting symptoms of hyperarousal.²¹ These findings highlight the necessity of further investigating factors contributing to individual differences in treatment outcome. Yet, this matter has been explored by only a few studies²² which were often limited by small sample sizes and thus lack the necessary statistical power to yield reliable findings. One IPD-MA on PM+ and SbS trials which is currently carried out (study protocol)²³ will hopefully shed light on moderators influencing treatment effects of such low-intensity interventions. However, results will be limited to PM+ and SbS trials only.

To paint a more complete picture, the present study aims to conduct an IPD-MA, in which datasets from separate randomised controlled trials (RCT) including both psychological and psychosocial interventions will be synthesised. IPD-MAs are considered the gold standard of statistical approaches when synthesising and analysing evidence from multiple studies.²⁴ By merging different IPD datasets with each other, a much larger sample size is reached than when looking at a single-study dataset, an advantage which allows for more complex analyses with statistical power and precision large enough to detect significant moderators of treatment effects and examine predictors of rare events such as adverse outcomes.²⁵ Additionally, the use of an IPD-MA will allow us to shed light also on moderators of treatment effects at client-level, something previous traditional meta-analyses using reported meta-data could not address as they are restricted to moderators at study-level.²⁶ Moreover, by including trials using specialised and low-threshold interventions, we will be able to examine whether interventions delivered by specialists and non-specialists differ in terms of treatment effects and moderators thereof. Specifically, for this IPD-MA, we aim to: (1) investigate treatment effects; (2) identify beneficiary, provider, intervention and study characteristics that moderate treatment outcome with regard to PTS symptom reduction among adult FDP receiving psychological and psychosocial interventions compared with controls receiving no intervention; and (3) extend the latter analysis to secondary outcomes including mental health outcomes other than PTS, non-response, attendance, attrition and adverse outcomes (see the ‘Statistical analysis’ section for more details).

Methods and analysis

Eligibility criteria

We will include trials that (1) used an RCT study design including (2) adult (≥ 18 years) FDP (ie, refugees, asylum seekers, or internally displaced persons, as defined by United Nations High Commissioner for Refugees²⁷) receiving (3) psychological and psychosocial interventions (eg, specific interventions such as CBT, low-intensity interventions such as PM+, or guided (eg, SbS) or unguided self-help programmes) or (4) a control condition without intervention (ie, no treatment, waiting-list, or case-as-usual), and which (5) assess PTS symptoms as outcome. Trials which included only a subsample of individuals with a forced-displacement background will be still included in this IPD-MA, if the target sample in the dataset can be identified.

Identification and selection of studies

We conducted a systematic literature search in the databases Medline, PsycINFO, PTSDpubs, Cochrane and Embase using search terms related to the population (ie, FDP), intervention (ie, psychological and psychosocial interventions), mental health outcomes (ie, general distress, PTS, depression or anxiety), and study design (ie, RCT). The search terms were identified through researchers and clinicians from the field; however, the target population was not consulted. The time range was not specified. Inclusion of studies were restricted to the following languages: English, German, French, Spanish, Portuguese and Dutch. Additionally, we searched the bibliographies and citations of 29 reviews and meta-analyses related to the topic. This search for relevant records provided by newly published reviews and meta-analytic work will be repeated before conducting the analyses. Their references, the detailed search syntax and the full search strings of each database can be seen here: https://osf.io/cbw3q/?view_only=2c42dff3c25a440cbd5a833e29e35c0b. The full search strategy is included in the online supplemental file. Before conducting any analyses, we will add the citations and bibliographies of all included articles to the screening process.

First, titles and abstracts of retrieved records will be screened independently by two raters to identify studies that potentially meet the inclusion criteria outlined above. Second, the full texts of these potentially eligible studies will be retrieved and independently assessed for eligibility by the same raters. Any disagreement between raters will be resolved through discussion with a senior rater where necessary. Retrieved records will be evaluated throughout the review process with the software COVIDENCE (https://www.covidence.org/).

Data collection, extraction and preparation

Authors of relevant trials identified in the selection process will be contacted to request anonymised data of their studies, that is, IPD including, but not limited to, the following variables: beneficiaries’ sociodemographic (eg, education), migratory (eg, time spent in host country), and clinical characteristics (eg, trauma history) and providers’ (eg, degree of training), intervention (eg, format), and study characteristics (eg, study setting). According to Polanin,²⁸ the success to obtain IPD from authors is moderate (ie, 58% success rate). In order to incentivise authors to share their data, we will offer two coauthorships per trial and contact all authors of each article at least three times, as suggested by Ventresca et al.²⁹

After gathering all primary datasets of the eligible studies, automated data quality checks for IPD will be run and data accuracy will be checked by comparing the frequencies of sociodemographic and clinical variables, as well as their mean scores and SD of continuous scales. Inconsistencies (eg, extreme values or discrepancies between the reported values and the delivered data) will be discussed and clarified with the authors of the primary trials. After confirming the accuracy of each dataset, we will first synchronise variables of interest to the same scale or categorical order and then merge the data into one large IPD meta-analytic dataset. If variables were assessed by several measures, the method with the highest quality standard will be selected (eg, clinical interviews will be favoured over self-report measures). Finally, outcome measures will be standardised by converting them to z-scores for each trial separately if multiple measures had been used for the same outcome (according to the procedure previously used by Karyotaki et al³⁰).

Quality assessment

The quality of included studies will be checked by two independent raters using the Revised Cochrane tool (RoB2.0) for assessing risk of bias in RCT.³¹ This tool assesses several domains including bias from the randomisation process, deviations from intended interventions and measurement of the outcome. Two bias categories, that is, ‘bias from missing outcome data’ and ‘selection from the reported result’, will not be assessed with the RoB tool. Instead, multiple imputation will be used to account for missing outcome data. The bias category ‘selection of the reported result’ is not applicable for IPD-MA as we will have access to the full datasets of all included studies. Each item will be evaluated regarding its risk resulting in a low or high risk of bias judgement per domain. Authors will be contacted in case of unclear items.

Statistical analysis

As PTSD is the most prevalent mental disorder in FDP,⁵ the primary outcome will be PTS symptoms assessed at post intervention (PT; that is, immediately after treatment) and follow-up (FU; at any later time). However, in order to paint a more complete picture, we will run analyses with secondary outcomes including positive mental health outcomes (eg, well-being), psychopathology (eg, depression), disability, functioning, and quality of life at PT and FU assessments, as well as adverse outcomes, attendance, attrition and treatment non-response. Moderator variables at client-level will depend on available IPD provided by the authors and will be included as moderators in the analyses if they are represented by at least three studies. Moderator variables at study-level will be extracted from the published manuscript and will consist of variables such as region where study was conducted (ie, low-/middle-income vs high-income countries), time of assessments and quality of study (assessed in the risk-of-bias quality assessment). In order to examine differences in treatment effects, we will include type of intervention (ie, low-intensity interventions vs specialised therapy) as a moderator in the analyses. Before running any main analyses (see below), we will first test all assumptions necessary for linear regression models using the R package “DHARMa” (https://cran.r-project.org/web/packages/DHARMa/vignettes/DHARMa.html).

The analyses will be conducted according to the intention-to-treat principle, that is, all randomised participants will be included in the analyses regardless of rationale for exclusion. Multiple imputation per trial will be conducted using 100 imputations through the mvn method in STATA software, StataCorp, as recommended by Graham et al.³² To estimate the missing values, complete baseline variables will be used (eg, PTS symptom levels at baseline, age, gender, etc). To assess the difference between imputed and complete values, we will conduct a sensitivity analysis using complete cases only. For the primary analyses, we will use the one-stage approach with IPD. Additionally, to compare effects of both type of trials, that is, those that provided IPD and those that did not, aggregate data meta-analyes using a two-stage approach including all IPD (transformed) and available meta-data from study reports will be conducted. This is particularly advisable when a large proportion of authors did not share their datasets.^{33 34} Results from both the one-stage and two-stage approach will be compared and discrepancies will be discussed.³⁵ As we will run several analyses with different outcome variables, we will correct for multiple testing (ie, Bonferroni adjusted p values) for analyses including secondary outcome variables. Analyses of the one-stage approach will be conducted using the STATA software (https://www.stata.com/), while all analyses of the two-stage approach and assumptions tests will be performed using the statistical program R (https://www.r-project.org/).

One-stage approach: analysis of IPD (primary analyses)

To investigate treatment effects of psychological and psychosocial interventions, we will perform a multilevel mixed-effects linear regression model with a random effect for each trial and fixed effects for intervention condition (treatment vs control) and severity of PTS symptoms at baseline. The severity of PTS symptoms at PT and FU will be used as the dependent variable. To identify moderators of treatment effects, we will add an interaction between each potential moderator and PTS outcome into the multilevel mixed-effects linear regression model. This procedure will be repeated for all aforementioned secondary outcome variables.

Two-stage approach: analysis of aggregate data (secondary analyses)

To investigate treatment effects, we will first calculate effect sizes for each trial separately and then compare them across studies by running aggregate data meta-analyses including both, trials providing IPD and studies providing meta-data only. Thus, we will run multivariate meta-analyses with standardised mean differences (ie, Hedges g³⁶) estimating the differences in PTS outcomes between participants in the intervention vs control group. We will use a random-effects model estimated by restricted maximum likelihood accounting for differences in trials.^{37 38} In order to identify moderators of treatment effects, we will first run several multiple linear regression models, including intervention condition (treatment vs control) and all potential moderators as independent variables and change in PTS symptom scores from baseline to PT and FU assessments as dependent variables for each trial separately. The obtained standardised regression coefficients for the interaction effect between intervention condition and each potential moderator will then be used as dependent variables when running several multivariate regression models with a random effect controlling for trial for each moderator separately. This procedure will be repeated for all secondary outcome variables mentioned above.

Heterogeneity (two-stage approach)

To quantify variation among studies we will conduct analyses of heterogeneity by using Cochran’s Q, prediction intervals and I² statistic.^39–41 I² is a measure which quantifies the proportion of observed heterogeneity representing the difference between effects sizes that are not due to sampling error but to differences in, for example, the populations or measures that are studied. It ranges from 0% to 100% including increments of 0%, 25%, 50% and 75%, indicating no, low, moderate and high heterogeneity, respectively.³⁹

Publication bias (two-stage approach)

We will assess publication bias by creating ‘contour-enhanced funnel plots’ for a visual evaluation of asymmetry⁴² and applying the ‘trim and fill’ method.⁴³

Certainty of evidence

To evaluate the confidence in evidence, we will apply the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) methodology for the primary outcome measure.⁴⁴

Patient and public involvement

None.

Ethics and dissemination

We issued a clarification of responsibility for which the local ethic committee of the canton of Zurich, Switzerland, confirmed that this IPD-MA does not require ethical approval (Req-2022–00496). Only anonymised datasets will be requested from authors. With signing our data transfer agreement, authors warrant that the provided data had been legally obtained and all necessary informed consents for the transfer to and use by a third party had been secured. The results will be published in international peer-reviewed journals.

Current status

The literature search as well as the screening of titles and abstracts and the full-text review has been partially conducted for this IPD-MA. The systematic literature search in the aforementioned databases had been carried out on 12 January 2022 and will be updated prior to conducting the analyses. This project is expected to be completed by December 2025.

Ethics statements

Patient consent for publication

Not applicable.