Evaluation of competency-driven training for facilitators delivering a psychological intervention for children in Lebanon: a proof-of-concept study

Abstract

Aims

The mounting evidence for effective delivery of psychological interventions by non-specialists in low- and middle-income settings has led to a rapid expansion of mental health and psychosocial support trainings globally. As such, there is a demand for strategies on how to train and implement these services to attain adequate quality. This study aims to evaluate the added value of a competency-driven approach to training of facilitators for a group intervention for children with severe emotional distress in Lebanon.

Methods

In a controlled before and after study, 24 trainees were randomly allocated to participate in either a competency-driven training (CDT) or training-as-usual (TAU) (1 : 1) for a psychological intervention for children with severe emotional distress. We assessed the change in demonstrated competencies, using standardised role-plays, before and after the training. Measures included the 13-item Working with children-Assessment of Competencies Tool (WeACT), the 15-item ENhancing Assessment of Common Therapeutic factors (ENACT) and the 6-item Group facilitation: Assessment of Competencies Tool (GroupACT). The trainer in the experimental arm used pre-training and during training competency assessment scores to make real-time adjustment to training delivery. Due to COVID-19 pandemic restrictions, all activities were done remotely.

Results

CDT resulted in significantly better outcomes on increasing competencies on the WeACT (repeated measures analysis of variance; F_{(1, 22)} = 6.49, p < 0.018) and on the GroupACT (Mann–Whitney U = 22, p < 0.003), though not statistically significant on the ENACT. There is no significant between-group difference on the reduction of harmful behaviours, mainly because both forms of training appear equally successful in eliminating such behaviours.

Conclusions

This proof-of-concept study demonstrates the potential of CDT, using standardised assessment of trainee competencies, to contribute to better training outcomes without extending the duration of training. CDT can result in up to 18% greater increase in adequate competency, when compared to TAU. The study also yields recommendations for further enhancing the benefits of competency-driven strategies. A fully powered trial is needed to confirm these findings.

Introduction

With increasing evidence for effectiveness of psychological interventions implemented by non-specialists (Singla et al., 2017; Cuijpers et al., 2018; Barbui et al., 2020), there is a need to demonstrate how task-shifted evidence-based treatments can be scaled-up to reduce the gap between need and availability of mental health care (i.e. treatment gap), especially in low- and middle-income countries (LMICs) (Patel et al., 2018). Efforts to reduce the treatment gap typically rely on a non-specialist workforce to increase the availability of care. In addition to availability of services, the consistency and quality of service-provision is essential to achieve the desired outcomes (Jordans and Kohrt, 2020).

An essential feature of quality of care is provider competence, defined as ‘the observable ability of a person, integrating knowledge, skills, and attitudes in their performance of tasks. Competencies are durable, trainable and, through the expression of behaviors, measurable’ (Mills et al., 2020). Especially for non-specialist providers in LMICs, there are no mechanisms in place to systematically monitor if providers achieve minimally adequate competency to effectively and safely deliver interventions (Kohrt et al., 2020). A recent systematic review evaluating quality assessments in adolescent mental health services concluded that most studies did not have a clear framework for measuring or defining quality (Quinlan-Davidson et al., 2021). To respond to this challenge the World Health Organization (WHO) has developed the Ensuring Quality in Psychological Support (EQUIP) platform (https://equipcompetency.org/en-gb) to facilitate competency assessment and competency-driven training (CDT) approaches (Kohrt et al., 2020). The platform includes measures that have been developed and evaluated in LMICs, such as the ENhancing Assessment of Common Therapeutic factors (ENACT) in Nepal (Kohrt et al., 2015a), and the Working with children-Assessment of Competencies Tool (WeACT) in Palestine (Jordans et al., 2021). This platform and measures can be used by governments and non-governmental organisations to assess whether providers have attained sufficient competencies to provide care, as well as by trainers and supervisors to tailor training and supervision content to the needs of individual or groups of providers to address gaps in competencies. The use of common competency assessment tools through a platform such as EQUIP may also lead to greater consistency of training quality across organisations and contexts.

CDT includes assessment of, and feedback on, the skills and competencies of participants before, during and after a training programme. This will guide the trainer in tailor-making the training (and subsequent supervision) to what is most needed for the group of participants (collectively and as individuals). CDT approaches have already demonstrated positive outcomes in mental health, for example a suicide prevention training (La Guardia et al., 2019), and in low-resource settings in other areas of health (cf. Ameh et al., 2016; La Guardia et al., 2019). The aim of this study was to evaluate the outcomes of a CDT for improving the quality of facilitators of a psychological treatment for young adolescents with emotional distress compared to training-as-usual (TAU). We hypothesised a CDT to outperform TAU in terms of levels of competencies obtained post-training.

Methods

Context

This proof-of-concept study was conducted in Lebanon. The Syrian crisis has had profound impacts on Lebanon, making it the second-largest refugee-hosting country, according to UNHCR Global Trends for 2021, and the host for the largest number of refugees from Syria (UNHCR, 2021). In an already suffering economy, the impact of war in Syria has added to increasing economic, social, demographic, political and security challenges.

Design

The current analysis is based on the EQUIP study in Lebanon which was a controlled before and after study (registration no. NCT04704362). The objective was to evaluate the outcomes of a CDT in improving the quality of facilitators of a psychological treatment for young adolescents with emotional distress (Early Adolescents Skills for Emotions – EASE) (Dawson et al., 2019), when compared to TAU. The subject of study therefore was the delivery of training, rather than the psychological intervention. Training participants (n = 24) were allocated equally (1 : 1) over the two study arms (CDT and TAU) using simple random sampling. Randomisation sequence was computer generated by an off-site statistician (GKG). Training and competency assessments were originally designed for in-person implementation. However, due to COVID-19 all training and competency assessments were done remotely, and implementation of the intervention was not possible.

Instruments

Facilitator competencies were assessed using the EQUIP tools based on observations by a trained competency rater. The rater was selected from a pool of five. Scoring was done using standardised competency tools, for working with children as the primary aim of EASE (WeACT), for working with adults as EASE also works with the caregivers of enrolled children (ENACT) and for working with groups as EASE is a group intervention (GroupACT). For each of the tools scoring was based on: (i) four levels of competency and (ii) attributes, which are distinct observable behaviours that serve to operationalise each of the four levels for each competency item. All of the tools are available through the online EQUIP platform (https://equipcompetency.org/en-gb).

WeACT

The WeACT aims to assess common competencies of service providers working with children and adolescents, and has been developed for and can be applied to interventions from different sectors (mental health and psychosocial support, child protection and education) (Jordans et al., 2021). The WeACT consists of 13 competency items, that can be scored following a four-tiered response system; level 1 (potential harm), level 2 (absence of some basic competencies), level 3 (all basic competencies) and level 4 (all basic + advanced competencies). We excluded one item from the original WeACT (i.e. collaboration with caregivers and other actors) because of the overlap with the ENACT. A previous study of the WeACT in Palestine has demonstrated positive results on reliability and utility (Jordans et al., 2021).

ENACT

For the assessment of competencies of facilitators delivering psychosocial and mental health care to adults, the ENACT was used (Kohrt et al., 2015a). The version used in this study consists of 15 competency items with the same four levels of appraisal as mentioned above. The ENACT has been extensively tested and used (Kohrt et al., 2015b, 2018).

GroupACT

For assessment of group facilitation competencies, the GroupACT was used. The GroupACT consist of eight items to assess facilitators' competencies in group-based mental health and psychosocial support interventions (Pedersen et al., 2021). For the purpose of this study we used a six item version. It follows the same structure and scoring system as the WeACT and ENACT.

Sample

For this study, 24 participants were selected to the training (CDT or TAU). Participants were recruited based on: (i) prior experience working with children, but with no specialist prior mental health experience or education and (ii) aged 18 years or above. The group of participants consisted of 15 females (62.5%) and 9 males (37.5%), mean age 26.4 (s.d. = 6.5) and with a mean of 6.0 (s.d. = 3.7) years of work experience. Recruitment was done through advertisement, focusing on Beirut and Mount Lebanon. No power analysis was conducted, as this was a proof-of-concept study.

Study conditions

Participants received either CDT or TAU of the intervention EASE. EASE is a WHO-developed transdiagnostic intervention for children (10–14 years) with severe emotional distress (Dawson et al., 2019). The intervention is designed to be delivered by trained non-specialised facilitators and is delivered using a group format. EASE comprises seven group sessions for children incorporating evidence-based techniques such as psychoeducation, stress management, behavioural activation and problem solving. Moreover, there are three group sessions for caregivers of enrolled children, including strategies such as psychoeducation, active listening, stress management, praise and caregiver self-care.

Participants in both arms received an 8-day training, including basic counselling skills, delivering EASE (child and caregiver sessions), skills in group facilitation and facilitator self-care. The training was based on the course previously developed for a randomised controlled trial (RCT) evaluating the effectiveness of EASE in Lebanon (Brown et al., 2019). Training content was the same in the two study arms. The difference between the arms was that in the CDT arm the trainer: (i) utilised the pre-training competency assessment outcome scores of the group to inform the delivery and dosage of training content and as feedback to participants, and (ii) utilised in-training competency assessment outcome scores to help trainers determine what competencies require additional attention in the remaining time and provide tailor-made feedback. See Table 1 for the differences between the two trainings.

Table 1.

Differences in training between study arms

	TAU	CDT
Pre-training evaluation	A structured multi-competency role-play of common factors is administered to participants and independently rated by someone external to the training. Scores are not provided to the trainer and not provided to the trainees.	A structured multi-competency role-play of common factors is administered to participants and independently rated by someone external to the training. Scores are provided to the trainer, who may then tailor training design or content to areas of strengths or weaknesses of the trainee cohort. Scores are also provided to the CDT trainees at the beginning of the training on an individual basis and group basis.
During the training	Training EASE helpers is delivered as the organisation would normally for the given intervention.	EQUIP common factor training modules can be incorporated into the training. Results from pre-training assessment and from single-competency role-plays used throughout can be employed to determine what common factors EQUIP modules are incorporated into the training. Throughout the training, competency assessment tools are used to check on skill development and areas for greater attention for individuals and the entire training group. Both single- and multi-competency role-plays and the regular intervention training practice are used. Peers are trained in giving feedback using the competency tools, and feedback is provided to the trainee by peers and trainer.
Post-training evaluation	Multi-competency role-plays of common factors and for treatment specific skills are administered to participants and independently rated. Scores are not provided to the trainer or supervisor and the TAU trainees.	Multi-competency role-plays of common factors and for treatment specific skills are administered to participants and independently rated. Scores are provided to the trainer and supervisor, who may then adapt upcoming supervision to areas of strength or weaknesses of the trainee cohort (either individual trainees or the whole group).

Due to COVID-19 restrictions, delivery of training was adapted to be delivered in an online format, using live conference calls. The training in each study arm was provided by a different trainer; both WHO-trained EASE master-trainers (MG, SC), with one trainer (SC) receiving an additional briefing on how to run CDT.

Procedures

All 24 training participants (CDT or TAU) were assessed for demonstrated competencies using standardised role-plays before (T0) and after the training (T1) using the WeACT, ENACT and GroupACT (see Table 1). Assessments (T0 and T1) were conducted by a trained rater observing all role-plays (n = 48 in total). Multiple role-play scripts were developed to ensure that all competencies (for children, groups and adults) being assessed were prompted for and could be demonstrated by the participants, and to ensure variation between the T0 and T1 role-plays. The use of role-plays for ENACT and WeACT assessments has been extensively tested in Nepal, Uganda, Liberia, Pakistan, Ethiopia, South Africa and Palestine. The role-play approach was shown to be feasible and acceptable in these settings (Kohrt et al., 2015a; Jordans et al., 2021), also because role-plays are commonly used as a component of training on psychological interventions (Ottman et al., 2020).

In this study we trained actors from a local theatre group to perform the live role-plays with the trainees. Competency assessments were adapted to an online role-play, where participants delivered components of the intervention to the trained actors via a live video conference call. The trained rater joined the same conference call to rate the role-play. The trained rater was selected from a competency rater training wherein five raters (all >2 years of experience in facilitating mental health interventions for children) received a 3-day training in rating competencies using the EQUIP platform. The rater with the highest compatibility to the intended scores that were specified for the structured video-taped role-plays was selected for the current study. Raters and actors were kept blind to participants' group membership.

Analyses

The competency assessment scores were analysed at different levels: (i) all the competency levels combined; (ii) only the harmful levels; (iii) only the competent levels and (iv) only the potentially harmful attributes, as well as only the helpful attributes.

For analysis (i) the scores 1 to 4 were summed across all competency items into a total score for each tool (WeACT, ENACT, GroupACT), and the means of the total scores were compared between TAU v. CDT (between groups) as well as T0 v. T1 (within groups). For analysis (ii) we did the same for the total count of scores 1 (harmful level). Equally for analysis (iii) for the total count of scores 3 (all basic competencies) combined with 4 (all basic + advanced competencies). Finally, we examined the scores at the attribute level (iv) by dividing them into potentially harmful attributes (level 1) and helpful attributes (levels 3 and 4). For each tool, the number of harmful and helpful attributes was summed across all competency items into a total of ‘harmful attributes’ score, and the number of helpful attributes was summed across all competency items into a total of ‘helpful attributes’ score, and we ran the same comparisons. We considered analyses (ii) and (iii) to be the primary outcome, and (i) and (iv) as exploratory outcomes.

All scores at the different analysis levels were checked for assumptions of normality and homogeneity of variance. When assumptions were met, repeated measures analyses of variance were conducted with the mean scores as the dependent variable, time (T0 v. T1) as the within-group variable, group (TAU v. CDT) as the between-group variable, a time × group interaction, with the latter as the effect of interest and gender as a covariate. Where assumptions were not met, separate non-parametric tests were conducted for differences between groups (Mann–Whitney U test) and within groups (Wilcoxon signed rank test). Throughout the paper we will adopt a significance threshold of p < 0.05.

Ethics

The study was approved by the Ethics Review Committee from Saint Joseph Université (SJU), Beirut, Lebanon and by the World Health Organization ethics review process, approved on 1 October 2019 (#CEHDF_1490). All study participants provided informed written consent to participate in the study.

Results

All competency levels combined

Assumption checks indicated that the total scores of the GroupACT were positively skewed and did not have equal variances across groups. Therefore, separate non-parametric tests were done. Table 2 shows the mean total scores of all levels combined for each of the tools. The WeACT and ENACT showed an increase in total scores from T0 to T1, F_{(1, 21)} = 27.1, p < 0.001, η_p² = 0.56, and overall the CDT group showed higher total scores than the TAU group, F_{(1, 21)} = 8.6, p < 0.01, η_p² = 0.29. However, the interaction effect of time and group was not significant, F_{(1, 22)} = 2.99, p = 0.10. The results of GroupACT can be interpreted as an interaction effect with a significant T0 − T1 difference for the CDT group, Wilcoxon signed rank test Z = −2.14, p < 0.05, but not the TAU group, Z = −0.36, p = 0.72, and a significant difference between the groups after the training, Mann–Whitney U = 17.5, p = 0.001, but not before the training, U = 72, p = 0.99.

Table 2.

Means and s.d.s of the total scores across competency levels for the three tools

Tool	CDT (n = 12)		TAU (n = 12)		Mdiff (s.e.)
	T0	T1	T0	T1
WeACT	36.5 (4.6)	41.1 (2.7)	34.8 (3.2)	36.4 (4.5)	4.7 (1.5)
GroupACT	12.2 (1.9)	14.3 (1.9)	12.1 (1.7)	11.9 (0.7)	2.4 (0.6)***
ENACT	33.1 (4.3)	36.3 (2.3)	29.8 (3.5)	34.2 (1.4)	2.1 (0.8)

CDT, competency-driven training; TAU, training-as-usual; WeACT, Working with children-Assessment of Competencies Tool; ENACT, ENhancing Assessment of Common Therapeutic factors; GroupACT, Group facilitation: Assessment of Competencies Tool; M_diff, mean difference at endline; s.e., standard error.

*p < 0.05, **p < 0.01, ***p < 0.001.

Only competent levels

The assumptions were not met for GroupACT scores, which were analysed with separate non-parametric tests. Table 3 shows the mean total counts of only the competent levels (levels 3 and 4) of each tool. The WeACT showed a clear interaction effect in competent scores, as expected with CDT showing a greater increase after the training, F_{(1, 22)} = 6.49, p = 0.018, η_p² = 0.23. The ENACT only showed significant effects of time and group in the expected direction with higher scores after the training compared to before, F_{(1, 22)} = 13.5, p = 0.001, η_p² = 0.38, and the CDT group showed higher scores than the TAU group, F_{(1, 23)} = 9.1, p = 0.006, η_p² = 0.29. The GroupACT showed only a significant difference between the groups after the training (Mann–Whitney U = 22, p = 0.003).

Table 3.

Means and s.d.s of the total counts of only the competent levels (3, 4) for the three tools

Tool	CDT (n = 12)		TAU (n = 12)		Mdiff (s.e.)
	T0	T1	T0	T1
WeACT	6.5 (1.9)	9.5 (1.5)	5.9 (0.9)	6.6 (2.2)	2.9 (0.8)*
GroupACT	1.1 (0.9)	1.8 (1.1)	1 (0.9)	0.3 (0.7)	1.5 (0.4)**
ENACT	4.2 (1.9)	5.1 (1.7)	2.1 (1.6)	4.1 (0.9)	1.0 (0.6)

CDT, competency-driven training; TAU, training-as-usual; WeACT, Working with children-Assessment of Competencies Tool; ENACT. ENhancing Assessment of Common Therapeutic factors; GroupACT, Group facilitation: Assessment of Competencies Tool; M_diff, mean difference at endline; s.e., standard error.

*p < 0.05, **p < 0.01, ***p < 0.001.

We demonstrated that CDT outperformed TAU, with a 17% increased improvement of competencies for trainees preparing to work with children (i.e. WeACT); and an 18% increase in the competencies of participants to work with groups (i.e. GroupACT); but a decrease of 6% in competencies for working with adults (i.e. ENACT) (see Fig. 1).

Fig. 1.

Change in proportion of items with adequate competency, and the between-group difference. WeACT, Working with children-Assessment of Competencies Tool; ENACT, ENhancing Assessment of Common Therapeutic factors; GroupACT, Group facilitation: Assessment of Competencies Tool; CDT, competency-driven training (n = 12); TAU, training-as-usual (n = 12). ^ABetween-group difference in pre to post change of percentage of skills at competency; *competency decreased from 13 to 4% pre- to post-training.

Only harmful levels

Because harmful scores did not occur frequently, the total counts at both timepoints and of all the tools were positively skewed and analysed with non-parametric tests. The WeACT showed a non-significant decrease in harmful scores from T0 to T1, for both the CDT and TAU groups which did not differ from each other (all p-values between 0.059 and 0.362). For the ENACT both CDT and TAU groups showed a significant decrease in harmful scores from T0 to T1 (CDT Wilcoxon signed rank test Z = −2.26, p = 0.024 and TAU Z = −2.83, p = 0.005). The groups did not differ significantly from each other on total number of harmful scores, neither before (Mann–Whitney U = 53, p = 0.253) nor after the training (U = 64.5, p = 0.633). For GroupACT only the CDT group showed a significant decrease in harmful scores, Wilcoxon signed rank test Z = −2.66, p = 0.008, however the number of harmful scores was not significantly different from the TAU group, neither before (Mann–Whitney U = 58, p = 0.351) nor after the training (U = 47, p = 0.080).

Potentially harmful and helpful attributes

The mean total scores at the attribute levels are shown in Table 4. The harmful attributes did not meet the assumptions and were analysed with non-parametric tests. The TAU group showed a significant decrease in harmful attributes after the training only on the ENACT (Wilcoxon signed rank test Z = −2.69, p = 0.007), while the CDT group showed a significant decrease in harmful attributes on both the ENACT (Wilcoxon signed rank test Z = −2.26, p = 0.02) and the GroupACT (Wilcoxon signed rank test Z = −2.66, p = 0.008). None of the between groups differences were significant (all p-values between 0.160 and 0.932). Regarding the helpful attributes, the results showed a significant increase over time for both the CDT and TAU groups on the WeACT, F_{(1, 22)} = 19.06, p < 0.001, η_p² = 0.46, and the ENACT, F_{(1, 22)} = 10.54, p = 0.004, η_p² = 0.32, but this effect did not differ between the groups on both tools. However, for the GroupACT, there was a significant interaction effect with an increase in helpful attributes in the CDT group and a slight decrease in the TAU group, F_{(1, 22)} = 5.78, p = 0.025, η_p² = 0.21.

Table 4.

Means and s.d.s of the total scores for the harmful and helpful attributes for the three tools

Tool	Attributes	CDT (n = 12)		TAU (n = 12)		Mdiff (s.e.)
		T0	T1	T0	T1
WeACT	Harmful	0.5 (0.9)	0.0 (0.0)	0.8 (1.0)	0.7 (2.0)	−0.7 (0.4)
	Helpful	31.0 (6.4)	38.2 (4.2)	26.2 (6.5)	30.3 (6.6)	7.9 (2.3)**
GroupACT	Harmful	1.4 (0.9)	0.2 (0.4)	1.1 (0.8)	0.6 (0.7)	−0.4 (0.2)
	Helpful	5.3 (2.9)	8.0 (3.8)	5.0 (2.9)	3.7 (1.9)	4.3 (1.2)**
ENACT	Harmful	2.2 (1.8)	0.7 (0.6)	3.1 (2.3)	0.8 (1.0)	−0.1 (0.3)
	Helpful	21.4 (4.9)	25.3 (4.5)	16.3 (4.4)	20.1 (3.1)	5.2 (1.6)**

CDT, competency-driven training; TAU, training-as-usual; WeACT, Working with children-Assessment of Competencies Tool; ENACT, ENhancing Assessment of Common Therapeutic factors; GroupACT, Group facilitation: Assessment of Competencies Tool; M_diff, mean difference at endline; s.e., standard error.

*p < 0.05, **p < 0.01, ***p < 0.001.

Discussion

The main objective of this proof-of-concept study is the between-group comparison between a CDT and a TAU (both remotely delivered) for a psychological intervention for children with severe emotional distress. The primary indicators of success are an increase in the mean number of items scored at adequate competence (levels 3 and 4 on the assessment tools), and a reduction in the mean score of the harmful behaviours (level 1 on the assessment tool) on completion of training. This study shows the potential added value of adopting a competency-driven approach to training non-specialist facilitators of mental health interventions, with up to 18% more improvement in demonstrated competencies post training. These results can be considered as proof-of-concept that requires further investigation to confirm effectiveness.

CDT outperforms TAU on the child-specific competencies (WeACT) and group-specific competencies (GroupACT), though not on adult-specific competencies (ENACT). These findings are in line with expectations given that the focus of the training was primarily for a group intervention for children, with only a small component focusing on adults (i.e. sessions with caregivers of the included children). The absence of improvement in the ENACT can also be explained by the lower pre-training competency in the control arm, allowing for more room for improvement. The results on the reduction of harmful behaviours show that both groups reduced – to near absence – harmful behaviours at post-training. This suggests that both versions of training were equally successful in changing potentially harmful behaviours (most pronounced on the ENACT), which explains the non-significance when comparing the groups based on the harmful level scores. Regardless of between-group differences, for ethical reasons it is important to note that using the competency assessment tools we are able to check that before starting to work with distressed children, none of the participants exhibits potentially harmful behaviours.

While we see improvement in competencies following training, it is also important to note that post-training there is still a significant proportion of items that are not assessed as adequate competency obtained. Hardly any items are assessed as potentially harmful, still one could question whether competency levels achieved in this training are sufficient to offer an intervention of high enough quality to help adolescent with emotional problems. This is an issue that plagues the field because so few studies systematically evaluate the competency to assure that psychological interventions can be safely and effectively delivered by non-specialists (van Ginneken et al., 2013; Singla et al., 2017). Because of this gap, it is vital that tools which capture helpful treatment skills and potentially harmful behaviours are used consistently in the field of global mental health (Jordans and Kohrt, 2020). Our current study aims to contribute to providing more systematic reporting of competency levels achieved, with the ultimate goal that by measuring competency routinely, the quality and safety of care in global mental health will be improved (Kohrt et al., 2020).

The remaining results (i.e. analyses of mean total scores, and of mean number of attributes scored) serve mainly descriptive reasons. The intended use of the competency tools is not to work with total scores. Mean frequency scores are not meant to be used as the primary outcome indicator because this can obscure the movement participants make between the competency levels, which is important information for demonstrating minimal adequacy on each of the competencies. Similarly, the attributes that form part of the competency assessment tools aid raters to determine the level of competency and for trainers to provide feedback during CDT, but mean total attributes scores are not meant to be used as the primary outcome indicator. Still, the results that the CDT group outperformed TAU on improvement in helpful attributes on the GroupACT are further support for the potential benefits of CDT.

These are promising findings for the added value of CDT of mental health and psychosocial support interventions in LMICs. The limited number of mental health professionals and limited budget available for mental health services in LMICs has resulted in a push for brief trainings of non-specialists who will deliver psychological interventions. It is salient that with the same limited resources, we can optimise the impact of training by following a competency-driven approach that uses standardised assessments before and during the training. It therefore has the potential to significantly contribute to better quality, more consistent and cost-effective mental health services in low-resource settings. The approach can be expanded to competency-based supervision (Falender and Shafranske, 2007), during the intervention implementation phase. Supervisee competencies can be determined through in-session observation or self-assessment using the standardised competency assessments tool, and used as input for supervision session, much like the assessments and subsequent feedback used in the CDT described in this study.

So what is the added value of a competency-driven approach? A competencies perspective argues that we cannot assume competencies are attained following training. Rather, there is a need for trainees to be able to explicitly demonstrate competence (Falender and Shafranske, 2012). Including results of standardised assessment of trainee competencies before and during training, which allows for: (i) tailor-made feedback, one of the key instruments of teaching, by identifying with areas that require improvement (Falender and Shafranske, 2012); (ii) increased attunement between trainer and trainee on what are concrete and observable expectations; (iii) more clarity on a benchmark for ‘readiness to start implementing’ and (iv) potentially greater consistency in the quality of intervention services being provided within and across organisations and contexts.

The current study is compatible with other initiatives that aim to augment the quality and impact of training for, and provision of, mental health services. For example, a recent pilot cluster RCT in Nepal shows that collaborating with people with lived experience as part of training primary health workers in integrating mental health services has the potential to result in reduced stigma and improve competence for diagnostic accuracy (Kohrt et al., 2021). These initiatives are in line with a need to shift the global mental health research agenda to evaluate strategies on how evidence-based interventions can best be implemented (Proctor et al., 2009), and how we can go about maintaining quality of care when services get scaled up (Jordans and Kohrt, 2020).

This focus on quality of care is important, because it is an essential predictor to adolescents seeking, receiving and continuing mental health care (Quinlan-Davidson et al., 2021). Initiatives to improve quality of health services have identified standards, such as health literacy, community support, adolescent participation, facility characteristics, non-discrimination and provider competencies (WHO, 2015). Provider competence is highlighted as a key standard (Quinlan-Davidson et al., 2021), with evidence demonstrating that competencies such as establishing provider–client relationship and alliance result in more positive outcomes (Gondek et al., 2017). While a recent meta-analysis failed to demonstrate the relationship between intervention-specific competencies and outcomes of psychotherapy for children and adolescents, it does suggest that non-specific competencies may be a more important factor (Collyer et al., 2020). This is in line with the focus of the measures that were used in the current study, where the WeACT, GroupACT and ENACT all aim to assess common-factors competencies. Moreover, regarding the hypothesised association between quality of care with improvements in clients mental health outcomes, it is salient to note that numerous studies have failed to show a consistent association (Barber et al., 2007; Rakovshik and McManus, 2010; Collyer et al., 2020). In a prior review, we have shown that gaps in methodological rigour on this topic have contributed to the limited ability to explore this relationship (Ottman et al., 2020). One of the main gaps has been failure to systematically measure competency outcomes of trainings using a standardised tool across programmes. Therefore, this study is a step forward in rigorously evaluated observable treatment competencies, as well as providing initial findings that the approach towards training impacts the competency level achieved.

This study is not without limitations. First, the study was implemented in the midst of the COVID-19 pandemic, which meant all activities needed to be done remotely. While challenging, we are pleased we managed to still implement the study. It did demonstrate that remote implementation of the competency framework appears feasible. At the same time, it may have impacted delivery of training or assessment of competencies, albeit equally in both study arms. Second, as mentioned above, given that this was not a RCT and that the study was underpowered due to the small sample size we cannot draw definitive conclusions about the effectiveness of a competency-driven approach to training compared to TAU. It is for that reason that we are now planning a fully-powered RCT. Third, though unavoidable, having different trainers in both study arms may have influenced results.

As this study was part of the EQUIP initiative, it also yielded multiple recommendations to further enhance the promotion of CDT. For example, the EQUIP digital platform now includes visualisations for trainers to easily capture harmful behaviours at the individual trainee level and the competencies rated as a level 1 ‘potentially harmful’ score. Based on lessons learned, the EQUIP platform also added training modules on how to conduct CDT and how to adapt manualised trainings into competency-driven approaches. A module on how to give and receive feedback using the EQUIP platform competency results was developed. As well, a Foundational Helping Skills curriculum that covers common factor skills such as those in WeACT and ENACT, was developed to help in remediation for competencies requiring additional training to reach safe and effective competency levels (Watts et al., 2021). With expansion of these tools and resources, there is potential for even greater benefits to support competency-driven strategies compared to standard training approaches.

Conclusion

With this study we demonstrate the potential of CDT, using standardised assessment of trainee competencies, to contribute to better training outcomes, without extending the duration of training. It holds promise to contribute to greater quality and accountability of psychological treatments implemented by non-specialists in LMICs. Further research is needed to test the effectiveness of such approaches in an RCT, as well as the application of a competency-driven approach to other aspects of the service delivery pathways, such as during supervision.

Financial support

Funding for the WHO EQUIP initiative is provided by USAID. We are grateful to the individuals who participated in the studies in each of the EQUIP sites and to all members of the EQUIP team for their dedication, hard work and insights. The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated. BAK and MJDJ receive funding from the U.S. National Institute of Mental Health (R01MH120649).

Ethical standards

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2000.

Availability of data and materials

The data supporting the findings of this study can be obtained from the corresponding author.

Conflict of interest

No conflict of interest to be reported.