Step-by-step: Feasibility randomised controlled trial of a mobile-based intervention for depression among populations affected by adversity in Lebanon

Abstract

Background

E-mental health interventions may help to bridge the mental health treatment gap. Evidence on their effectiveness is compelling in high-income countries. Not enough evidence has been generated on their use with communities affected by adversity in low- and middle-income countries. The World Health Organization (WHO), the National Mental Health Programme (NMMP) at Ministry of Public Health (MoPH) in Lebanon and other partners have adapted a WHO intervention called Step-by-Step for use with Lebanese and displaced people living in Lebanon. Step-by-Step is a minimally guided, internet-based intervention for adults with depression. In this study, a feasibility randomised controlled trial (RCT) and a qualitative process evaluation were conducted to explore the feasibility and the acceptability of the research methods, and the intervention, in preparation for two fully powered trials to assess the effectiveness and cost-effectiveness of Step-by-Step in Lebanon.

Method

Participants were recruited through social media. Inclusion criteria were: being able to understand and speak Arabic or English; access to an internet connected device; aged over 18; living in Lebanon; scores above cut-off on the Patient Health Questionnaire and the WHO Disability Assessment Schedule 2.0. Participants were randomly assigned to the intervention or enhanced care as usual. They completed post-assessments eight weeks after baseline, and follow-up assessments another three months later. Primary outcomes were depression and level of functioning, secondary outcomes were anxiety, post-traumatic stress, and well-being. Qualitative interviews were conducted to evaluate the feasibility and acceptability of the research procedures and the intervention.

Results

A total of N = 138 participants, including 33 Syrians, were recruited and randomised into two equal groups. The dropout rate was higher in the control group (73% post- and 82% follow-up assessment) than in the intervention group (63% post- and 72% follow-up assessment). The intervention was perceived as relevant, acceptable and beneficial to those who completed it. Suggestions were made to further adapt the content and to make the intervention more engaging. Statistical analyses were conducted despite the small sample size. Complete cases analysis showed a statistically significant symptom reduction in depression, anxiety, disability, and post-traumatic stress, and statistically significant improvement in well-being and functioning. Intention-to-treat analysis revealed non-significant effects.

Conclusion

The research design, methods and procedures are feasible and acceptable in the context of Lebanon and can be applied in the RCTs. Preliminary findings suggest that Step-by-Step may be effective in reducing symptoms of depression and anxiety and improving functioning and well-being.

Highlights

• •An e-intervention was tested among people affected by adversities in Lebanon.
• •The study showed that the intervention and methods can be applied in full RCTs.
• •The intervention may reduce symptoms of depression and improve functioning.

In low- and middle-income countries (LMICs), the discrepancy between high prevalence rates of mental disorders and limited availability of mental health services is well documented (Alonso et al., 2018; Thornicroft et al., 2017). Particularly high prevalence rates of common mental disorders (CMDs) such as depression, anxiety and post-traumatic stress disorder (PTSD) have been reported by refugees and other people affected by violent conflict (Charlson et al., 2016; Turrini et al., 2017). Innovative approaches are needed in order to increase coverage of mental health treatment in LMICs, and particularly for refugee populations.

In recent years, the World Health Organization (WHO) has developed a series of potentially scalable interventions designed for low-resource and humanitarian settings that can be delivered by specialists, non-specialists or in self-help format (WHO, 2017). In the past decade, guided self-help has increasingly been delivered through the Internet as a potential way to reach more people. In high-income countries, guided e-mental health interventions have been shown to be as effective as similar interventions delivered face-to-face (Andersson et al., 2019; Cuijpers et al., 2019) and have been found to reduce symptoms of mental disorders in routine care (Andersson and Titov, 2014). An increasing number of trials have been conducted in LMICs, with promising effect sizes (Arjadi et al., 2015; Fu et al., 2020; Naslund et al., 2017). Despite the potential of such e-mental health interventions, challenges regarding uptake and adherence remain (Torous et al., 2018; Watson et al., 2018).

As part of work to develop and publish evidence-based psychological interventions as public goods, WHO in collaboration with the National Mental Health Programme (NMHP) at the Ministry of Public Health (MoPH) in Lebanon, Freie Universität Berlin, and University of Zurich developed an internet-based guided self-help intervention for adults with depression called Step-by-Step (Carswell et al., 2018). This was subsequently culturally adapted by WHO, the NMHP-MoPH in Lebanon, and University of Zurich, for use in Lebanon. Lebanon is a small middle-income country facing political and economic turmoil, with high need for mental health services. The country hosts around 1 million registered Syrian displaced people (UNHCR, 2021), in addition to approximately 0.5 million unregistered refugees, further overstretching the health care system. A nationally representative study among the Lebanese population showed lifetime prevalence of 12.6% for mood disorders, and 25.8% for anxiety (Karam et al., 2008). A recent survey study among Syrian displaced people revealed a point prevalence of moderate to severe depression symptoms of 22% (Naal et al., 2021). There is a large mental health treatment gap. In the nationally representative sample (which was done prior to the influx of Syrian people), only 11% of respondents with a diagnosis had received treatment, and treatment is often substantially delayed (Karam et al., 2006). The NMHP at MoPH integrated internet-based interventions into their strategy as a way to respond to this treatment gap (Lebanese Ministry of Public Health, 2015). A systematic review on adapting mental health intervention in Middle Eastern Arab countries showed that there is a need to partner and work within the local and cultural context, and to develop culturally appropriate treatments (Gearing et al., 2012). The process and results of culturally adapting Step-by-Step to different populations living in Lebanon are described in a separate paper (Abi Ramia et al., 2018).

Step-by-Step is a 5-week programme that uses a narrated story to teach skills and techniques for coping with symptoms of depression. The therapeutic components are based on the WHO guidelines for the treatment of mental disorders in LMIC (Dua et al., 2011), which include recommendations for cognitive-behavioural therapy (CBT) for depression. Step-by-Step includes the following CBT components: behavioural activation, psychoeducation, stress management, positive self-talk, strengthening social support and relapse prevention. Minimal guidance and motivation are provided remotely (i.e., through phone calls or messages), during weekly support contacts (max. 15 min), by trained non-specialists called “e-helpers” (Carswell et al., 2018).

The first version of Step-by-Step was a website designed by the University of Zurich that could only be accessed through a web-browser. An uncontrolled pilot study of this first version was conducted to test the feasibility and acceptability of Step-by-Step (Harper Shehadeh et al., 2020). This uncontrolled pilot study showed relatively high drop-out rates, as only 25% of participants completed all five sessions. Upon completion of the uncontrolled pilot study, and based on a qualitative process evaluation, an iOS, Android and web app version was developed and user tested at Freie Universität Berlin (Burchert et al., 2019). The application includes more interactive features to increase engagement. The effectiveness and cost-effectiveness of Step-by-Step will be tested in two parallel, fully-powered RCTs among Syrian displaced people (trial 1, N = 568) and Lebanese and other people residing in Lebanon (trial 2, N = 568) (van't Hof et al., 2021).

In preparation of the two RCTs that will be conducted in Lebanon, a feasibility RCT was implemented with the aim to evaluate the feasibility of the research procedures, and the acceptability of the app version of Step-by-Step. The present paper reports on the preparation, methods and results of this feasibility RCT, and data related to the acceptability of the intervention.

1. Methods

1.1. Design

In this study, a two-arm, single-blind randomised controlled trial (RCT) was conducted to test the research methods and procedures in Lebanon. Participants were randomly assigned to Step-by-Step or enhanced care as usual (ECAU, see below). The allocation was 1:1 using permuted block randomization.

1.2. Intervention

Step-by-Step is delivered via an iOS and Android app that can be used offline (after initial download) on mobile devices, or via a web app that can be accessed via web browser. Participants in the intervention condition received access to Step-by-Step, along with e-helper support. Step-by-Step encompasses five sessions plus a short introduction, with a recommendation that users complete approximately one session per week over five weeks. Access to sessions was given consecutively, i.e., a session had to be completed to activate the subsequent one. Upon completion of the last session, users could still access Step-by-Step but did not receive e-helper support anymore.

Users in the control condition received one information session within the app or website. This session consisted of basic psychoeducation on depression and anxiety, and a list of primary health care centres in all areas of Lebanon trained in detection and management of mental health problems. The text for the psycho-educational messages was taken from the first session of Step-by-Step to make sure the information provided is identical.

1.2.1. E-helper training

A team of six e-helpers received three days training on the intervention and the study, followed by three half-days of role plays. E-helpers are non-specialists who offer minimal guidance to Step-by-Step users (up to a maximum of 15 min weekly). E-helper support for users is provided through phone call or chat messaging based on the users' preference; it includes technical and motivational help, the support of coping with distress if needed and referral of potentially high-risk cases to specialist care or hospitalisation. Templates to support users via phone or chat were developed and translated to Arabic. In collaboration with the legal advisor, the clinical supervisor, and the policy and advocacy coordinator at the NMHP-MoPH in Lebanon, procedures for dealing with potential high-risk situations were developed, which included imminent risk of suicide, child protection, sexual and gender-based violence, and potential harm to other adult people. The protocols provided clear steps and templates for e-helpers, to make sure users in potential high-risk situation receive optimal support. Potential high-risk situations were repeatedly trained in role plays during weekly clinical supervision.

1.3. Recruitment strategy

The uncontrolled pilot had shown that 63% of the participants were recruited through social media (Facebook and WhatsApp), while outreach, in-person or through services was not that efficient (Harper Shehadeh et al., 2020). Nevertheless, only 19% of the participants in the uncontrolled pilot study had been Syrians. To ensure recruitment of Syrians and improve recruitment rates more generally for this feasibility RCT, community participatory methods were used to inform the strategy, including five focus group discussions with a total of 38 Syrian volunteers and key informants from the United Nations High Commissioner for Refugees (UNHCR) office in Lebanon. A communication and advertising company was contracted to develop a social media recruitment strategy along with new recruitment material. Consequently, a range of posts were developed and distributed through the NMHP Facebook and Instagram pages, which included static pictures, animations, and a recruitment video that represented all socio-economic classes and religions in Lebanon. The new posts provided clarifications and reassurances around concerns and misconceptions raised during the focus group discussions. Since social media had been more effective in attracting Lebanese rather than Syrians, new outreach methods had to be explored. A partnership was established with UNHCR to support dissemination of the posts to the Syrian community via their messaging strategy over WhatsApp and through their outreach volunteers. Organizations in the Mental Health and Psychosocial Support (MHPSS) taskforce were also approached to help disseminate the project among their beneficiaries, and a group of social workers from the International Medical Corps (IMC) have been trained to introduce the project in their outreach activities.

1.4. Participants and procedures

Participants were Syrian displaced people, Lebanese and other people residing in Lebanon. Inclusion and exclusion criteria were: a) being able to understand and speak English or Arabic language; b) access to an internet connected device; c) age of 18 or older, d) residing in Lebanon, e) a score of 10 or higher on the Patient Health Questionnaire (PHQ-9, Kroenke and Spitzer, 2002), and f) a score of 16 or higher on the WHO Disability Assessment Schedule (WHODAS 2.0, Ustun et al., 2010) for functional impairment. People reporting imminent risk of suicide (i.e., answering “yes” to the question “In the past month, have you had serious thoughts or a plan to end your life?”) were excluded from the study.

From social media posts, interested people could directly access the app or web-version of Step-by-Step. The app started with the study information and informed consent, followed by screening questions. Applicants not meeting inclusion criteria received an on-screen message thanking them for their interest in the study, and explaining that they could not participate in this study at this point. In case of imminent risk of suicide, applicants received a message encouraging them to seek help at a health centre, and a refund of transportation costs if required. The message also provided the contact details of the national Life-Line, a specialised telephone service that provides emotional support and suicide risk assessment, intervention and referral to specialised mental health care.

Participants who met the inclusion criteria were asked to provide contact information for reminders when the post- and follow-up assessments were due (i.e., phone number or e-mail address). Thereafter, they completed the baseline assessments within the app. Upon completion of the assessments, they were informed about their group allocation and could either start using Step-by-Step (intervention condition), or read the basic information (control condition). Users in the intervention condition were asked about their preferred method of contact for e-helper support (i.e., phone calls or chat messages within the Step-by-Step platform). Eight weeks after baseline assessments, users were contacted for post-assessments, and three months later they received the invitation for the follow-up assessments.

1.5. Measures

Primary outcome measures were the PHQ-9 for depression (Kroenke and Spitzer, 2002) and the WHODAS 2.0 (Ustun et al., 2010) for functional impairment. Secondary outcome measures were the General Anxiety Disorder Scale (GAD-7, Spitzer et al., 2006), the six item version of the PTSD Checklist for DSM-5 (PCL-5, Price et al., 2016), the WHO-5 Wellbeing Index (Bech et al., 2003), and the Psychological Outcome Profiles instrument (PSYCHLOPS, Ashworth et al., 2004), a self-report problem rating scale. Data on socio-demographic information (gender, age, education, marital status and work status) was collected through questions A1-A5 of the 12 item self-report version of the WHODAS 2.0. During the course of the intervention, the PHQ-4 (Ishihara et al., 2019; Kroenke et al., 2009) was used to monitor depressive symptoms.

To examine the acceptability of the intervention, we analysed usage data, i.e., the percentage of participants who opened the sessions and the exercises, completed the exercises, and played the audio-exercises.

1.6. Qualitative process evaluation

A qualitative process evaluation was conducted with 21 key informant interviews with participants and the project team. Participants were selected following a quota sampling method from those who completed the post-assessments, and were evenly divided between the treatment and the ECAU groups. Representation of different nationalities and genders was ensured. Phone interviews were carried out by the project coordinator (JAR) with the participants and an independent researcher (RAH) interviewed the project team to avoid bias. Attempts to approach participants who dropped out were made via the application messaging platform and by phone, but were unsuccessful as these participants were hard to reach.

A semi-structured interview guide assessed general perceptions of the intervention content and app features, intervention adherence, burden of assessments, rapport with e-helpers, as well as the supervision model (in interviews with the study team). Interviews were recorded, translated and transcribed to English and a thematic analysis was conducted. Results were cross-checked between the two data collectors (JAR and RAH). Results guided further improvement of the intervention and research methods in preparation for the definitive trials.

1.7. Statistical analyses

Since this was a feasibility RCT, the study was not sufficiently powered to detect treatment effects. However, the dataset was used to test whether the planned statistical analyses can be conducted. For this purpose, both intention-to-treat analysis (including all randomised participants) and completer analyses (per protocol, PP) were carried out. To measure comparisons at baseline between the two treatment groups, t-tests were conducted for continuous variables and Chi-squared test for categorical ones. In addition, descriptive analyses were conducted for usage data (i.e., the use of the different intervention elements).

To estimate treatment effects, the mean difference between the two treatment arms at baseline, post-intervention, and three-month follow-up was first determined. Initial effectiveness analyses were derived based on complete-case analysis. Complete-case estimates and effect sizes are obtained by estimating an ordinary least squares regression with the post-assessment outcome as dependent variable and as independent variables an intervention dummy variable indicating whether a participant was assigned to use the app or not and the baseline assessment (i.e., pre-score) as a covariate for enhanced precision. Then, the treatment effect was estimated based on intention-to-treat (ITT), using regression estimation models with the principal predictor being treatment assignment status. Missing outcome observations for participants were imputed using multiple imputation (MI), exploiting pre-scores and a set of pre-specified background characteristics (gender, age, education, and severity of symptoms). Imputation was done through a multivariate normal regression using an iterative Markov Chain Monte Carlo method based on initial treatment assignment. This MI considers missing data to be Missing at Random (MAR). In addition, bias concerns as a result of potential non-random missing outcome observations (i.e. MNAR) were addressed by estimating Lee bounds (Lee, 2009).

The aforementioned treatment effect analyses were performed for both primary outcome measures PHQ-9 and WHODAS 2.0. For explorative purposes, the same analyses were also carried out for the following clinical outcomes measured at each assessment time: anxiety (GAD-7), well-being (WHO5), posttraumatic stress reactions (PCL-5, eight-items version) and self-identified symptoms (PSYCHLOPS).

1.8. Ethics

Ethical approval was received locally from the Saint-Joseph's University (USJ) in Beirut (Protocol: CEHDF862) and from the WHO Ethical Review Committee (Version 7; Protocol ID: ERC.0002797).

2. Results

2.1. Sample description

A total of N = 138 participants were randomised, of which n = 33 were Syrians (see Fig. 1). The low number of Syrians was due to a far higher number of Lebanese signing up for the study. In the first week n = 143 Lebanese were recruited for the study, compared to 17 Syrians. To avoid over-recruitment, the social media campaign was stopped and additional Syrians were recruited via an established UNHCR messaging strategy over WhatsApp.

Fig. 1.

Flow chart.

The number of participants in the control condition was slightly higher due to a larger drop-out in the intervention group that was caused by a technical issue which occurred in the first week of the feasibility RCT. This technical problem was solved within a week. The socio-demographic information is presented in Table 1. The majority of participants (63.8%) had heard about Step-by-Step over the web (i.e., Facebook, Instagram, etc.), and the rest from health care workers (9.4%), friends (8%), family members (3.6%), posters (2.9%) or other channels (12.3%).

Table 1.

Demographic information.

Demographic information	Intervention group (n = 67)	Control group (n = 71)	Group comparison
Age, M (SD)	28.2 (7.90)	26.4 (7.95)	t(132) = 0.746, p = .19
Age range	18–50	18–50
Gender, n (%)			χ2(1) = 0.003, p = .96
Female	45 (67.2)	48 (67.6)
Marital status, n (%)			χ2(4) = 1.12, p = .89
Never married	44 (65.7)	43 (60.6)
Married	19 (28.4)	22 (31.0)
Separated	1 (1.5)	3 (4.2)
Divorced	2 (3)	2 (2.8)
Widowed	1 (1.5)	1 (1.4)
Nationality, n (%)			χ2(3) = 5.32, p = .15
Lebanese	48 (71.6)	52 (73.2)
Syrian	15 (22.4)	18 (25.4)
Palestinian	4 (6)	0 (0)
No information	0 (0)	1 (1.4)
Education, n (%)			χ2(3) = 4.15, p = .25
Elementary	8 (11.9)	7 (9.9)
Secondary	10 (14.9)	17 (23.9)
University	41 (61.2)	44 (62.0)
Technical	8 (11.9)	3 (4.2)
Professional situation, n (%)			χ2(3) = 1.60, p = .66
Paid work	24 (35.8)	27 (38.0)
Unpaid work	10 (14.9)	9 (12.7)
Student	16 (23.9)	22 (31.0)
Unemployed	17 (25.4)	13 (18.3)

The lost-to-post-assessment rate (eight weeks after baseline assessment) on the PHQ-9 was 62.7% in the intervention and 73.2% in the control condition, the lost-to-follow-up rates (five months after baseline assessment) were 71.6% and 81.7%, respectively. The observed difference in dropout between the two conditions was not statistically significant. Missing observations were not systematically related to observed background characteristics or pre-scores (Prob > chi2 = 0.388, n = 138). The two groups did not significantly differ on sociodemographic characteristics or pre-scores (see Table 1). However, the higher attrition in the control group resulted in slightly lower average pre-scores on the primary outcome measures (i.e., PHQ-9 and WHODAS 2.0) for the control condition, relative to the intervention condition, when including complete cases only in the analysis.

2.2. Primary outcome measures

The reliability of the primary outcome measures at baseline assessment for the full sample was satisfactory with Cronbach's α = 0.73 on the PHQ-9 Cronbach's α = 0.83 on the WHODAS 2.0. Means and standard deviations are reported in Table 2. The complete case analysis at post-assessment revealed a statistically significant reduction of depressive symptoms as measured with the PHQ-9 (Cohen's d = 0.73; p = .009), and a statistically significant reduction in functional impairment measured with WHODAS 2.0 (Cohen's d = 0.57; p = .046). Whereas we do not have sufficient power to adequately assess whether results differ between both samples in this feasibility RCT, interaction effect analysis showed that the complete case treatment effect estimate was not statistically significantly different for Syrians than for Lebanese and other people living in Lebanon, on both the PHQ-9 (p = .546) and WHODAS 2.0 (p = .803).

Table 2.

Mean, standard deviation, and N for pre- and post-assessment divided by condition and assessment point, regression coefficients, effect sizes (complete cases) and Lee bounds.

Measure and group	Pre-assessment			Post-assessment (8 weeks)			Effect (within) Complete Cases	Complete Case		Multiple Imputation		Effect (between) Complete Cases	Lee bound (lower)	Lee bound (upper)
	M	SD	N	M	SD	N	d [95% CI]	b (SE)	p	b (SE)	p	d [95% CI]	b [95% CI]	b [95% CI]
PHQ-9
Intervention	16.52	4.49	67	7.56	4.98	25	−1.55 [−2.18 to −0.92]	−4.808 (1.75)	0.009	−4.64 (2.78)	0.105	−0.726 [−1.33 to −0.12]	−6.36 [−10.42 to −2.29]	−2.50 [−7.06–2.05]
Control	17.18	4.93	71	11.75	6.82	20
WHODAS
Intervention	32.90	8.43	67	26.26	8.71	23	−0.943 [−1.55 to −0.33]	−5.34 (2.59)	0.046	−7.29 (3.68)	0.053	−0.573 [−1.19–0.05]	−5.00 [−13.34–3.35]	1.20 [−6.59–9.00]
Control	32.25	9.31	71	27.95	9.34	19
WHO-5
Intervention	5.79	3.98	67	11.75	5.93	24	1.328 [0.70–1.95]	3.84 (1.59)	0.020	3.70 (2.38)	0.127	0.75 [0.12–1.38]	0.66 [−3.81–5.13]	6.44 [2.20–10.68]
Control	6.37	3.80	71	8.33	4.23	18
GAD-7
Intervention	14.99	3.99	67	8.13	5.41	23	−1.75 [−2.42 to −1.09]	−3.94 (1.80)	0.035	−3.35 (2.37)	0.164	−0.61 [−1.24–0.02]	−6.15 [−10.65 to −1.64]	−0.90 [−6.60–4.80]
Control	14.04	4.87	71	11.17	5.53	18
PCL-5
Intervention	21.45	5.93	67	14.33	6.72	24	−0.92 [−1.51 to −0.32]	−3.76 (2.05)	0.074	−3.76 (3.03)	0.223	−0.56 [−1.18–0.06]	−5.50 [−10.36 to −0.64]	0.86 [−4.31–6.03]
Control	20.23	7.36	71	16.56	6.00	18
PSYCHLOPS
Intervention	17.18	3.49	67	12.44	5.70	23	−1.02 [−1.63 to −0.40]	−4.31 (1.70)	0.016	−4.37 (2.57)	0.100	−0.73 [−1.38–0.08]	−6.72 [−12.14 to −1.30]	−1.13 [−5.12–2.86]
Control	17.09	3.04	71	16.12	4.47	17

Although this feasibility RCT was not sufficiently powered, an intention-to-treat analysis using multiple imputation was calculated, which revealed a non-significant between-group effect for the PHQ-9 (p = .105) and a marginally non-significant result on the WHODAS 2.0 (p = .053). Non-random attrition (i.e., higher drop-out rates in the control condition) is likely to bias the comparison between intervention and control group. Even though differential dropout between both conditions was not statistically significant, there was little statistical power to test this hypothesis. To investigate the extent in which differential dropout between conditions might bias the treatment effect estimates reported, we applied the Lee (2009) bounds strategy. Under the assumption of monotonicity, the true PHQ-effect is likely to be between −6.4 and − 2.5 [95% CI: −10.42; 2.05], and the true effect for the WHODAS 2.0 lies between −5.0 and 1.2 [95% CI: −13.34; 9.00].

Between-group effect sizes for symptom reduction between pre- and follow-up-assessments (i.e., three months after post-assessment, Table 3) were similar, with Cohen's d = 0.81 (p = .009) for the PHQ-9 and Cohen's d = 0.87 (p = .001) for WHODAS 2.0. Intention-to-treat analysis using multiple imputation showed a non-significant effect for the PHQ-9 (p = .405) and the WHODAS 2.0 (p = .222).

Table 3.

Mean, standard deviation, and N for follow-up assessment divided by condition, regression coefficients, effect sizes (complete cases, pre-follow-up), and Lee bounds.

Measure and group	Follow-up (Three month)			Effect (within) complete cases	Complete case		Multiple imputation		Effect (between) complete cases	Lee bound (lower)	Lee bound (upper)
	M	SD	N	d	b (SE)	p	b (SE)	p	d [95% CI]	b [95% CI]	b [95% CI]
PHQ-9
Intervention	8.33	5.31	18	−1.82 [−2.60 to −1.05]	−5.11 (1.83)	0.009	−5.15 (6.10)	0.405	−0.81 [−1.57 to −0.05]	−8.18 [−13.89 to −2.46]	−1.52 [−7.04–4.00]
Control	13.08	5.40	12
WHODAS
Intervention	25.10	10.45	19	−1.10 [−1.78 to −0.42]	−9.95 (2.70)	0.001	−8.06 (6.47)	0.222	−0.87 [−1.62 - -0.11]	−12.37 [−23.48 to −1.26]	1.89 [−10.02–13.80]
Control	29.92	10.05	12
WHO-5
Intervention	11.53	6.31	19	1.17 [0.48–1.85]	3.99 (2.00)	0.056	2.28 (5.09)	0.657	0.72 [−0.02–1.47]	−0.18 [−6.75–6.40]	8.30 [2.34–14.27]
Control	7.58	4.23	12
GAD-7
Intervention	8.32	6.22	19	−2.10 [−2.89 to −1.31]	−5.50 (1.92)	0.008	−5.24 (8.24)	0.531	−0.79 [−1.53 - -0.04]	−9.17 [−15.75 to −2.60]	0.02 [−6.93–6.96]
Control	11.92	4.80	12
PCL-5
Intervention	13.90	7.31	19	−1.23 [−1.92 to −0.53]	−5.99 (2.41)	0.019	−7.03 (9.23)	0.454	−0.76 [1.51 - -0.01]	−8.73 [−15.77 to −1.69]	1.02 [−6.35–8.39]
Control	17.75	6.97	12
PSYCHLOPS
Intervention	13.13	6.17	15	−1.07 [−1.84 to −0.31]	−5.23 (2.18)	0.023	−7.18 (9.05)	0.436	−0.79 [1.53 - -0.04]	−8.25 [−14.97 to −1.53]	0.17 [−5.39–5.73]
Control	17.00	3.16	11

2.3. Secondary outcome measures

The reliability of the secondary outcome measures at baseline assessment for the full sample was satisfactory with Cronbach's α = 0.836 on the WHO-5, α = 0.829 on the GAD-7, α = 0.869 on the PCL-5, and α = 0.742 on the PSYCHLOPS. Means and standard deviations are reported in Table 2. The complete case analysis revealed a statistically significant reduction of anxiety (GAD-7) between pre-and post-assessment (Cohen's d = 0.61; p = .035), a statistically significant improvement on self-defined problems as measured with the PSYCHLOPS (Cohen's d = 0.73; p = .016), and a significant improvement of well-being as measured with the WHO-5 (Cohen's d = 0.75; p = .020). The between-group effect for symptoms of post-traumatic stress PCL-5 was not statistically significant (Cohen's d = 0.56; p = .074). Again, an intention-to-treat analysis using multiple imputation was calculated despite the fact that this feasibility RCT was not fully powered. It revealed non-significant effects for secondary outcome measures. Results for post-assessments, including Lee (2009) bounds are presented in Table 2.

At follow-up, significant between-group effects were found for complete case analyses of the GAD-7 (Cohen's d = 0.79; p = .008), the PCL-5 (Cohen's d = 0.76; p = .019), and the PSYCHLOPS (Cohen's d = 0.79; p = .0.023). The effect for WHO-5 was marginally non-significant (Cohen's d = 0.72; p = .056). Intention-to-treat analysis showed non-significant effects for secondary outcome measures. Results for follow-up assessments, including Lee (2009) bounds are presented in Table 3.

2.4. Usage of step-by-step elements

Table 4 shows the usage of sessions and activities. The percentage of users who opened the sessions declined from 96% (Introduction) to 31% (Session 5). Between 70% and 80% of users who opened a session did at least one of the activities within that session. While planning an activity was done by most participants, a smaller percentage used the calendar in the app to schedule these activities for a particular day and time. With regard to stress management, the grounding exercise was used by more participants than the slow breathing exercise. Most users played the grounding exercise between one and four times, but a small percentage (13%) played it between 11 and 26 times.

Table 4.

Usage of sessions and activities.

Session/activity	n of participants who opened the session (% of participants in intervention condition)	n of participants who did the activity (% of those who opened the session)
Introduction	64 (96%)
Grounding		63 (98%)
Session 1	50 (75%)
Slow breathing		25 (50%)
Planning a small pleasant activity		40 (80%)
Scheduling the activity in calendar		27 (54%)
Session 2	35 (52%)
Planning a challenging activity		28 (80%)
Scheduling the challenging activity in calendar		6 (17%)
Session 3	30 (45%)
Kind words to oneself		21 (70%)
Problems faced with completing activities		13 (43%)
Potential solutions to these problems		10 (33%)
Session 4	23 (35%)
Social activity		17 (74%)
Session 5	21 (31%)
Potential warning signs for relapse		16 (76%)
Mood tracking	27 (40%)

2.5. Minimal guidance

Most users chose guidance through phone call (n = 34, 51%), five users chose support through the chat function, and two users used both contact methods. The other n = 26 (39%) participants did not respond to the e-helper contact. The mean number of contacts was 4.7, and 4.2 when excluding contact for potential high-risk situations. The mean total contact time per user was 78.5 min, and 54.7 min when excluding contacts regarding potential high-risk situations. This means that the maximal contact of 15 min per week was not exceeded. The mean time used per chat was higher (41 min) than per phone contact (11.4 min).

2.6. Qualitative process evaluation

Results of the qualitative process evaluation are presented here in summary as more details will be available in a forthcoming paper. Results showed that participants who completed the intervention regarded it as relevant, essential, empowering and relaxing, especially in a context where stigma of mental health disorders prevails. Nevertheless, it was deemed essential to further clarify the structure and purpose of the definitive RCT, as several participants in the control group did not fully grasp that they were part of a trial. Recommendations pertaining to the content of the intervention suggested to make it more engaging and appealing to a wider audience (e.g., by developing additional versions of the story featuring unmarried characters, or adding more context-specific challenges). Further suggestions to make the app more interactive included adding automated messages and push notifications, and summarizing the study information and consent or replacing them with animations as they were perceived to be too long. Reasons for drop-out were mainly technical issues or having other priorities and difficulties in life. The e-helper support model was also assessed to be effective and very helpful in increasing adherence and improving wellbeing.

3. Discussion

This study tested the feasibility and acceptability of the research procedures to inform two parallel and fully powered RCTs in Lebanon. These two parallel RCTs will evaluate the effectiveness and cost-effectiveness of Step-by-Step among (a) Syrian displaced people and (b) Lebanese and other people residing in Lebanon (van't Hof et al., 2021). There is a scarcity of RCTs testing internet-based interventions in low- and middle-income countries (Fu et al., 2020). These settings pose challenging circumstances to the implementation of a trial. With our paper, we aimed to provide insights on how a large, fully powered RCT can be implemented under such conditions. In addition to testing the research procedures, this study examined the acceptability of the app version of Step-by-Step.

This study revealed important insights for the fully powered RCTs. First, our study showed that social media is a very effective recruitment channel in Lebanon. This finding is congruent with other studies (Carmi and Zohar, 2014; Fletcher et al., 2019; Kayrouz et al., 2016; Ünlü Ince et al., 2014). Yet, the recruitment strategy needs further refinement to improve recruitment of Syrians, as the numbers recruited through social media were much lower (24% of participants were Syrians). Establishing partnerships with UNHCR, IMC and other non-governmental organizations from the MHPSS taskforce who work closely with Syrians in Lebanon is deemed crucial to address this issue. It is noteworthy that the community participatory approach adopted for the preparation of the recruitment material appeared a key factor for successful buy-in of the population.

A second insight concerns drop-out. In the intervention condition, 69% of users did not complete the intervention, and around 70% of users in the intervention and control condition did not complete the post- and follow-up assessments. There was a small difference in drop-out rates between conditions. This difference was not statistically significant, which could be due to the small sample size. In a systematic review of studies with internet-based interventions, Melville et al. (2010) found drop-out rates between 2 and 83%, with a weighted average of 31%. Considering that Step-by-Step is a guided intervention, it is important to examine reasons for drop-out, and to find potential means to increase adherence. Nevertheless, it was not possible to reach those who dropped out during the feasibility RCT to explore potential reasons for that. The technical problems encountered could partially explain the high drop-out rates. Also, results from the qualitative process evaluation provided suggestions that could improve adherence, e.g., making the story more realistic and exciting, and rendering the intervention more engaging by using notifications and motivating messages. For the definitive RCT, messaging will be tweaked to better explain the importance of participating in a research project.

The analysis of usage data also showed that most participants had used the stress management exercises that were presented at the start of the intervention. Easier exercises (e.g., writing down friendly words to oneself) were more frequently used than more complex exercises, such as writing down problems and solutions for completing the planned activities. These results suggest that easy-to-use exercises which do not require much cognitive effort are more attractive than more complex tasks. In addition, our results also showed that the time invested in support provided through chat was considerably higher than through telephone. Based on this result, we will change from (synchronous) chat to (asynchronous) messaging for the definitive RCT.

All other research procedures, i.e., randomization, reminders for post- and follow-up assessments, and response to potential (serious) adverse events, have shown to work well for the fully powered RCT. The qualitative process evaluation showed that onboarding and assessments was experienced as burdensome by some participants and that people did not grasp the idea of research very well. This is not unusual for RCTs, and might be particularly relevant in contexts where the concept of research is unfamiliar. Therefore, in the definitive RCTs, there will be more focus on explaining the concept of research and the assessments as clearly as possible to motivate users to participate and complete all assessments.

Although this was not a fully powered RCT, statistical analyses on the quantitative outcomes were conducted and important trends were found. Complete case analysis revealed a statistically significant reduction of depression, anxiety, and disability level at post-assessment, and a statistically significant increase in well-being. Statistically significant effects were found also in the complete case analysis at three-months follow-up for depression, anxiety, disability level, and PTSD symptoms. Only the group difference on the well-being measure (WHO-5) was smaller. Given the high drop-out rate in this study, conclusions have to be drawn with caution. Intention-to-treat analysis using multiple imputation revealed non-significant effects at post-assessment and follow-up, which was to be expected due to the small sample size and consequently insufficient power to detect the effects.

This study has several limitations. First the sample size was too small to analyse the data separately for each group, as will be done in the to fully powered RCTs. The main aim of this study was to evaluate the research procedures and the feasibility of the intervention, and not the statistical analysis. Second, the uncontrolled pilot study had already shown that participants wanted the story to be more engaging, and have additional stories about unmarried characters who do not have children (Harper Shehadeh et al., 2020). In the project's timeline, it was not possible to write these new stories before the start of the feasibility RCT. Therefore, the old version of the story was used, with the only difference that it had been divided into smaller parts to make it suitable for the app version. The new stories will be used for the definitive, fully-powered RCTs. Third, at the beginning of the feasibility RCT, a number of participants experienced technical issues with the newly developed app software. Some of these issues occurred under rare conditions and were therefore not identified in pre-trial software tests. Others were caused by limited mobile bandwidth and disconnections for which the software required further optimization. The technical issues were fixed during the feasibility RCT and e-helpers were perceived as “quick and professional” in assisting with these issues. However, participants and e-helpers mentioned that problems related to technical issues were frustrating and sometimes led to discontinuation.

Despite these limitations, the study showed that procedures of the RCT and the intervention were feasible and acceptable. Although drop-out rates were considerably high, the feedback received by participants was positive and revealed important suggestions to further improve the intervention. Our results indicate that Step-by-Step improves symptoms of psychological distress such as depression, anxiety, and post-traumatic stress among those who complete the intervention. Despite the high drop-out rates, the intervention may still have an impact on the lives of people who need help. From a public health perspective, taking into consideration the large number of people who do not have access to treatment in low-resource and humanitarian settings, the potential benefit of this intervention remains high. The two planned fully powered RCTs are needed to confirm the effectiveness of the intervention using intention-to-treat analysis. If proven effective, Step-by-Step can be delivered to various populations in different places of the world, thereby making an important contribution to narrowing the worldwide mental health treatment gap.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.