Practice-Driven Evaluation of a Multi-layered Psychosocial Care Package for Children in Areas of Armed Conflict

Abstract

Psychosocial and mental health service delivery frameworks for children in low-income countries are scarce. This paper presents a practice-driven evaluation of a multi-layered community-based care package in Burundi, Indonesia, Sri Lanka and Sudan, through a set of indicators; (a) perceived treatment gains; (b) treatment satisfaction; (c) therapist burden; (d) access to care; (e) care package costs. Across four settings (n = 29,292 children), beneficiaries reported high levels of client satisfaction and moderate post-treatment problem reductions. Service providers reported significant levels of distress related to service delivery. Cost analyses demonstrated mean cost per service user to vary from 3.46 to 17.32 € depending on country and specification of costs. The results suggest a multi-layered psychosocial care package appears feasible and satisfactory in reaching out to substantial populations of distressed children through different levels of care. Future replication should address therapist burden, cost reductions to increase sustainability and increase evidence for treatment efficacy.

Background

It is widely accepted that there is a vast gap in child mental health and psychosocial support provision in conflict-affected Low and Middle Income Countries (LAMIC), given the consistently higher levels of psychological distress and mental disorders identified in such settings (Belfer 2008; Patel et al. 2007). A recent systematic literature review demonstrated a lack of field-tested or researched intervention models for children in armed conflict (Jordans et al. 2009). Several publications have advocated comprehensive multi-layered care programs (Betancourt 2005; de Jong 2002; Fairbank et al. 2003; IASC 2007; Saltzman et al. 2003). A layered system of care entails provision of complementary supports for sub-populations depending on severity of mental health problems. In practice, however, most reported interventions for children in conflict-affected settings follow a single-intervention approach (Jordans et al. 2009). Multi-layered interventions are necessary because of the wide-ranging impact of war and the limited capacity of the informal and formal health care systems in resource-poor settings. A recent World Health Organization (WHO) rapport states that interventions should not be provided as freestanding activities, but delivered in a variety of packages and through different levels of a health system to provide a way of scaling up interventions in LAMIC (WHO 2008). Moreover, stepped care models have been introduced as an attempt to maximize efficiency of resource allocation in therapies with lower cost interventions implemented first and more costly interventions reserved for those insufficiently helped by the initial intervention (Haaga 2000). Lack of awareness by parents and children of mental health problems and services, lack of understanding, recognition and management by primary health care workers and teachers of mental health problems, and the stigma and discrimination related to mental health have been reported as primary barriers to child and adolescent mental health care in LAMIC (Tyano and Fleishman 2007).

Goldberg and Huxley (1980), in their classic filter-model, present an example of multi-layered pathways to mental health care. It demonstrates how potential patients are selected, through a set of filters, from one level of care to another. Following this combination of pathways and barriers to care, we present a care-utilization model for a community-based care package for children in war-affected areas (Burundi, Sudan, Sri Lanka and Indonesia). This package follows a public health approach including primary-, secondary- and tertiary-prevention interventions (de Jong 2002) and aimed at translating guidelines for complex emergencies (IASC 2007; Weiss et al. 2003) into a replicable delivery framework.

Figure 1 depicts utilization of the care package, including filter mechanisms for different levels of care. Level 1 comprises mental health promotion activities for the population at-large. Level 2 consists of peer-resilience group activities (ranging from recreational activities to peer group theme-centred discussions) to prevent healthy, albeit at-risk, populations to develop psychosocial problems. In order to reach as many children as possible in a systematic way the program targets school-going children. As not all children are school going, access constitutes the first filter. Level 3 targets subgroups of children within the school setting that present with elevated levels of psychosocial problems and who are subsequently offered to participate in a 15-session Classroom-Based Intervention (CBI) to reduce psychological distress (Macy et al. 2003). A primary screener was developed and used (see below) for early detection of children between level 2 and 3. Screening presents the second filter. Level 4 represents treatment of sub-groups or children with more severe psychological distress that require more focused individualized care (i.e. counselling or family/parental support) that can be delivered by training para-professionals (Ali et al. 2003). Filter 3 represents the detection, by para-professional CBI facilitators, of children in need of follow-up care due to inability to participate in CBI or sustained problems after termination of CBI, subsequently referred to counselling. Finally, a small group of children will require specialized psychiatric services to treat mental disorders. Mental health facilities in community settings in LAMIC and child and adolescent mental health professionals in specific are sparse (Patel et al. 2007), therefore the limited referral options present the fourth filter in the model. The program was funded by Plan Netherlands, a child-focused development agency concentrating on the promotion of child rights in low-income settings, and took place between September 2004 and August 2009. It was designed as a pilot project to develop a multi-layered psychosocial care response for children in areas of armed conflict, with the aim of sustaining the care package in the four countries and replicating it in other similar settings. For a detailed description on the content of the care delivery framework we refer to previous publications (see: www.psychosocialcarechildren.org).

Fig. 1.

Utilization and evaluation of care package

This paper provides a practice-driven evaluation of a multi-layered psychosocial care package for children in four conflict-affected countries. Evaluation of complex interventions that exist of several components of care present with methodological and feasibility challenges, as different levels of care require different indicators, and consequently tend to be marked by a paucity of high quality research (Campbell et al. 2000). We therefore employ a monitoring and evaluation framework, combining process evaluation methods and outcome methods (Hubbard and Miller 2004), applying the following criteria; (a) perceptions on treatment outcome by client, parents and service providers; (b) client and therapist treatment satisfaction; (c) therapist burden; (d) level of selection and access to care following the levels outlined above, and; (e) care package costs. The latter two criteria cover the entire care package and the first three criteria cover the core interventions (see Fig. 1).

Methods

Participants and Settings

We studied populations in four countries; Burundi, Sudan, Sri Lanka and Indonesia, all active or post-armed conflict settings. The Burundian sample contains children from 3 north-western provinces. The Republic of Burundi has been afflicted by killings and violence along ethnic and regional lines re-erupting in a civil war from 1993 until 2003, when a peace agreement was signed (Amnesty International 2004). The last rebel group, Palipehutu-FNL, joined the peace process in 2009, but the peace process is still fragile. The Sri Lankan sample consists of school-going children from 3 educational zones in and around Jaffna (Northern Sri Lanka). The origins of the violence in Sri Lanka, between the mainly Buddhist Sinhalese community and the mainly Hindu/Roman Catholic Tamil community, date back to the time of independence from Britain. In 1983, the Liberation Tigers of Tamil Eelam (LTTE) launched an armed struggle for a Tamil Homeland, due to perceived discrimination by the Sinhalese Government (Amnesty International 2006). The Indonesian sample consists of children from five sub-districts of Poso, Central Sulawesi, an area that has seen recurrent violence since 1998 as a result of hostility between Muslim and Protestant populations, though the conflict has its roots in wider economic and political shifts (Aragon 2001). Major hostilities lasted until the Malino accords in 2001, with continued incidences of violence ever since. The Sudanese sample consists of children in 3 Payams in Yei County, South Sudan. Since 1955, Sudan has been afflicted for more than 36 years by inter and intra-tribal and ethnical regional conflicts, caused by competition over meagre resources and power-positions. The civil war between the Islamist Central Government (consisting of Northern Sudanese elites) and peripheral areas represented by the Sudan People Liberation Movement/Army (SPLM/A) formally ended in a Comprehensive Peace Agreement in 2005 (Berghof Foundation for Peace Support 2006). All data collection procedures were consistent with the Declaration of Helsinki.

All interventionists were from the project countries and where possible hired from targeted communities. Training of the different interventionists were all conducted by local staff and varied in length depending on the service. Community resilience activities were conducted by Community Psychosocial Workers, a group of community volunteers who were typically trained for a period of 1–2 weeks. CBI facilitators received a training course of 3 weeks, which was designed and master-trained by the Center for Trauma Psychology from Boston, USA (further information available upon request). Finally, counsellors were trained following a skills-based training course over a period of 4 months, which was overseen by technical advisors of HealthNet TPO [MJ, WT] (Jordans et al. 2003). Average monthly remuneration for the interventionists varied between fees for CBI facilitators (45 € in Indonesia; 60 € in Sri Lanka; 55 € in Burundi, and 145 € in Sudan) and salaries for counselors (e.g. 154 € in Indonesia; 80 € in Sri Lanka; 180 € in Burundi, and 25 € in Sudan). Remunerations were compatible with local standards and organizational practices in each setting.

Applied Evaluation Indicators

Selection and Admittance to Different Levels of Care

One of the primary process indicators to evaluate a multi-layered care package is the level of detection and utilization of each level of care. The flow of clients through different levels of treatment displays feasibility and barriers of a care package. Throughout the project (2005 to mid-2008) the number of direct and indirect beneficiaries at each level of the care package (see Fig. 1) has been continuously monitored. The most structural detection mechanism involved primary screening to distinguish children with elevated psychosocial distress for optimal allocation of children to the group-based CBI (filter between levels 2 and 3 in the model above). The Child Psychosocial Distress Screener (CPDS) is a brief, context-specific and multi-informant primary screener developed for that purpose. A validation study has demonstrated the CPDS to have good concurrent validity (Area Under the Curve [AUC] = .81; optimal cut-off score >8) for detection of caseness for psychosocial care in Burundi (Jordans et al. 2008). The score range for the CPDS is between 0 and 14. As schools were the entry point of the program this is where primary screening took place. The CPDS was finalized by December 2006, therefore data presented below were collected between 2006 and mid-2008. This data present a baseline level of psychosocial distress experienced by children. In addition, we monitored the number of children referred for follow-up care as a result of secondary screening by the project’s service providers.

Treatment-Related Criteria

Treatment-related criteria include; (a) perception of treatment outcome; (b) treatment satisfaction; and (c) therapist burden. Continuous monitoring of services was integrated throughout the care delivery phase. Evaluation of treatment-related indicators was limited to level 3 and 4 interventions because of the level of standardization in implementation of these services. Children, service-providers and parents were asked to complete structured (combined quantitative and qualitative) questionnaires upon completion of the intervention, designed for the purpose of this program. The self-report questionnaires consisted of 4–9 items, with an approximate administration time of 5–7 min. The questionnaires were pilot-tested and adapted in the initial phase of the project. We evaluated perceived treatment outcome (i.e. levels of perceived problem reductions), mean client satisfaction (i.e. level of satisfaction of needs and satisfaction with services) and mean therapist burden (i.e. levels of distress as a result of providing care) for CBI and the counselling interventions. Two satisfaction questions were taken from the Client Satisfaction Questionnaire (Attkisson and Zwick 1982). Quantitative questions were scored on a 5-point response format. All questionnaires were verbally administered. Interviewers read the questions out loud and recorded respondents’ responses. This was done to control for the variable literacy aptitude of participants and because of respondents’ unfamiliarity with completing standardized questionnaires. Content analyses were conducted on qualitative data.

Cost Criteria

Mental health care in LAMIC is scarcely subjected to economic evaluation, yet cost analyses are important to assess efficiency, inform policy development regarding allocation of limited financial resources or to stimulate investment (Chisholm et al. 2000). We aimed to analyze the cost of developing and implementing a comprehensive care package in resource poor settings, in order to evaluate viability for future replication. Mean cost per service user was calculated by dividing the total sum of program development and intervention delivery costs (numerator) by the number of beneficiaries included in the care package (denominator). We used annual financial reports for 2005–2008 for the numerator data and the total reached beneficiaries for denominator data. Beneficiaries were defined as individual who received and visited services, regardless of outcome of services. We have made two types of cost calculations. To calculate actual costs of development and implementation of a comprehensive care system all costs, including project management, administrative and service providing staff, transport (including fuel, insurance and maintenance), capacity building of service providers, equipment, office costs, intervention and supervision costs were included. To calculate actual costs of sustaining a developed and downsized care package for children, all above costs except initial capacity building and counselling costs were included. Similarly, we made a distinction in calculating number of beneficiaries; (a) All children who received direct service delivery subsequent to screening (i.e. CBI, resilience groups and counselling), and; (b) Total beneficiaries, including above-mentioned children screened as well as children, parents and teachers reached through group psycho-education sessions. Cost ratio was calculated per country sample over a 3.5-year period.

The author(s) declare that there are no known conflicts of interests. All authors read and approved the final manuscript and certify responsibility for the paper.

Results

Selection and Admittance to Services

Over a period of 3.5 years 19,065 children were enrolled in the Classroom Based Intervention, 24,690 children in the child resilience groups, 1,705 children received counselling services and 571 children have been referred to more specialized services. Additionally, 25,723 parents and teachers have been reached through psycho-education groups. Breakdown of age groups and gender divisions per country and per intervention haven been summarized in Table 1.

Table 1.

Overview of beneficiaries

	Targets	Burundi	Indonesia	Sudan	Sri Lanka
Level 1	Psycho-education groupsa (N)	12,151	4,795	6,638	2,139
Level 2	Child Resilience Groups (N)	13,690	1,858	7,375	1,767
	Age (%)
	7–9	NA	39	17	18
	10–12		57	64	69
	13–15		4	19	13
	Gender (%)
	Male		48	56	51
	Female		52	44	49
Level 3	Classroom based intervention (N)	8,335	3,594	4,472	2,664
	Age (%)
	7–9	10	36	17	10
	10–12	52	61	81	52
	13–15	36	3	2	34
	Gender (%)
	Male	54	54	67	50
	Female	46	46	33	50
Level 4	Counsellingb (N)	731	150	548	276
	Age (%)
	7–9	8	24	4	18
	10–12	44	64	38	68
	13–15/19	45	12	58	14
	Gender (%)
	Female	53	37	66	47
	Male	47	63	34	53
Level 5	Referrals (N)	359	13	131	68

Note: NA not available

^aOnly parents and teachers

^bThis also includes parental care/family support provided by the counsellors. Total number of children screened is the total of the level 1 + level 2 beneficiaries

The above data provide an overview of the total number of children reached by the program. Data on levels of detection and admittance to levels 3–5 can be obtained from the screening data. CPDS data of 2.5 years provide baseline levels of experienced distress across the four countries (see Table 2). Comparable percentages of children were included in the group-based CBI intervention (41.1% in Burundi; 42.4% in Indonesia; 38.1% in Sudan; 42.4% in Sri Lanka). Upon secondary screening by CBI facilitators, the number of children internally referred to follow-up care by a counsellor is 6.6% for Burundi; 3.1% for Indonesia; 8.6% for Sudan and 10.4% for Sri Lanka. Subsequently, the numbers of children referred by counsellors to specialized care ranged between 0.1 and 2.9%.

Table 2.

Total scores and percentages of children indicated for treatment

	Burundi	Indonesia	Sudan	Sri Lanka
	N = 12,008	N = 3,770	N = 8,428	N = 5,086
CPDS Total score M (SD)	6.77 (2.91)	6.74 (2.67)	6.01 (4.85)	3.86 (2.67)
Indicated (%)	41.4	42.4	38.1	42.4
Referred to counsellorsa (%)	6.6	3.1	8.6	10.4
Referral to mental health professionalb (%)	2.9	0.3	0.1	1.3

Note: CPDS = child psychosocial distress screener

^aChildren referred after receiving CBI subsequent to screening outcomes; community referrals to counsellors therefore excluded

^bChildren referred after receiving counselling

Treatment Evaluation Classroom Based Intervention (CBI)

Most beneficiaries and service providers of CBI rated the service positively, among service providers 92% in Burundi, 89% in Indonesia, 98% in Sudan and 94% in Sri Lanka reported to be quite or very much satisfied. Among children 94, 97, 95 and 98%, respectively, reported to be quite or very appreciative of the service. Levels of perceived problem reductions were somewhat lower. Among children, 71% in Burundi, 48% in Indonesia, 78% in Sudan and 85% in Sri Lanka reported quite a bit or a lot of problem reductions. Among parents these percentages were 83, 87, 82 and 84%, respectively. The major changes that children (n = 11,595) reported are improved emotional wellbeing (i.e. reduced fear, sadness, anger and negative thoughts), as well as positive social and behavioural changes (i.e. increased patience, sociability, self-esteem, concentration, unity, tolerance). Service providers (n = 1,595) most commonly reported behavioural changes (i.e. reduced indiscipline, aggression, hyperactivity and increased obedience) and social changes (i.e. increased unity, peer interaction, trust, inclusion and collaboration). Parents and teachers (n = 4,660) similarly reported emotional (i.e. reduced fear, nightmares, depressed moods, anger) and behavioural changes (i.e. increased concentration, diligence, independence and reduced aggression). The perceived changes in obedience and respect were most commonly mentioned across the four settings, which respondents often related to improved school functioning (i.e. increased interest in school and study). In addition to these common categories, we found that children and parents/teachers in Sri Lanka and Indonesia commonly mentioned increased skills (i.e. improved drawing, increased creativity, knowledge and awareness of talents) and in Burundi reduced stress complaints. A commonly reported change in Indonesia was further the reduction of tensions and suspiciousness between children of different religions. Across the 4 countries, service providers experienced significant distress while conducting CBI groups (44% in Burundi; 52% in Indonesia; 35% in Sudan; 33% Sri Lanka, scored quite a bit or very much).

Treatment Evaluation Counselling

Most commonly presented intake problems among the children were; fears (23%), behavioral problems (i.e. aggression, theft, violation of rules; 17%), sleeping problems (16%), and learning difficulties (10%) in Burundi; school performance problems/learning difficulties (36%), aggression (32%), concentration problems (26%) and family conflict (22%) in Indonesia; worries and negative thought (68%), learning difficulties (53%), sadness (55%) and fears (46%) in Sri Lanka; social withdrawal and friendship formation difficulties (32%), behavioral problems (i.e. aggression, indiscipline, disrespect; 24%), poor concentration problems (17%), stress complaints (14%), sadness (12%) and low self-esteem (8%) in Sudan. Percentages do not add up to 100% as children could present with multiple intake problems.

Average clients’ satisfaction of needs after receiving counselling ranged from 3.43 to 4.31 (see Table 3). In other words, 63% in Burundi, 44% in Indonesia, 87% in Sudan and 43% in Sri Lanka reported to be quite a bit or very much satisfied. When comparing with rates of satisfaction with overall service, levels of high satisfaction were reversed for Burundi (32%) and Indonesia (84%), and comparable for Sudan (94%) and Sri Lanka (55%). Clients alike reported high levels of general improvement after counselling (95% in Burundi; 95% in Indonesia; 89% in Sudan; 89% in Sri Lanka). When asked for level of problem reduction, 72% in Burundi, 67% in Indonesia, 61% in Sudan and 30% in Sri Lanka reported to be quite a bit or very much reduced. In terms of reported changes, counselors in Burundi most frequently mentioned decreased fear (11%), decreased stigmatization (10%), reduced nightmares (6%), and reduced substance use (6%); in Indonesia reduced behavioral changes (i.e. aggression, obedience; 34%), anxiety and fear (17%), as well as improved school-performance (12%); in Sudan decreased bad thoughts (14%) and fear (12%), as well as improved sleep (17%) and communication (12%); in Sri Lanka decreased fear (44%), sadness (32%), naughtiness (28%), as well as increased school attendance (40%) and self-esteem (27%). Children’s most frequently reported changes were similar, with additional reporting of performance-oriented changes related to improved concentration, problem-solving, self-confidence and overall school functioning. Counselors experienced significant distress as a result of service provision (41% in Burundi; 47% in Indonesia; 5% in Sudan, and 28% in Sri Lanka, report quite a bit or very much distress). In Indonesia, with the smallest sample size (n = 106), we observed the highest average counsellor distress score (3.47), and the lowest average perceived client progress score (3.14). The average counsellor distress score per country was lower than that of CBI facilitators. Moreover, it appeared that lower client progress was associated with increased therapist burden for counsellors, as well as for group facilitators (Table 4).

Table 3.

Evaluation data CBI

Item	Country	Sample (N)	Average	Response range
Facilitator distress	Burundi	571	3.18	5 = Very much
	Indonesia	515	3.43	4 = Quite a bit
	Sudan	262	3.03	3 = A little
	Sri Lanka	299	3.05	2 = Hardly
				1 = Not at all
Facilitator satisfaction	Burundi	571	4.24
	Indonesia	515	4.16
	Sudan	258	4.60
	Sri Lanka	297	4.35
Child satisfaction	Burundi	7,230	4.61
	Indonesia	3,306	4.77
	Sudan	1,873	4.64
	Sri Lanka	2,034	4.87
Problem reduction according to children	Burundi	7,230	3.87	1 = Not al all reduced
	Indonesia	3,270	3.22	2 = Hardly reduced
	Sudan	1,795	4.04	3 = A little reduced
	Sri Lanka	2,034	4.35	4 = Quite a bit reduced
				5 = A lot reduced
Problem reduction according to parents & teachers	Burundi	502	3.87
	Indonesia	3,638	4.00
	Sudan	200	4.18
	Sri Lanka	609	4.07

Table 4.

Evaluation data counselling

Item	Country	Sample (N)	Average	Response range
Counsellor perception of client progress	Burundi	270	3.28	1 = Deterioration
	Indonesia	106	3.14	2 = No change
	Sudan	169	3.38	3 = Some improvement
	Sri Lanka	162	3.56	4 = Much improvement
Counsellor distress	Burundi	270	2.89	5 = Very much
	Indonesia	106	3.47	4 = Quite a bit
	Sudan	165	2.20	3 = A little
	Sri Lanka	153	2.70	2 = Hardly
				1 = Not at all
Child satisfaction of needs	Burundi	256	3.44	5 = Almost all needs met
	Indonesia	55	3.43	4 = Most needs met
	Sudan	158	4.31	3 = Some needs met
	Sri Lanka	101	3.49	2 = Only few needs met
				1 = None needs met
Child satisfaction of service	Burundi	256	3.20	5 = Yes, definitely
	Indonesia	55	4.25	4 = Yes, I think so
	Sudan	158	4.27	3 = Maybe
	Sri Lanka	101	3.89	2 = No, I do not think so
				1 = No definitely not
Child general improvement	Burundi	256	2.95	1 = Deterioration
	Indonesia	55	3.25	2 = No change
	Sudan	139	3.80	3 = Some improvement
	Sri Lanka	101	3.55	4 = Much improvement
Child perception of extent of problem reduction	Burundi	256	3.65	1 = Not at all reduced
	Indonesia	55	3.72	2 = Not really reduced
	Sudan	148	3.71	3 = A little reduced
	Sri Lanka	101	3.33	4 = Quite reduced
				5 = Much reduced

Cost Calculations

Cost analyses demonstrated that the development and implementation of the entire care package (combination of 3-tiered intervention structure; i.e. psycho-education, non-therapeutic resilience groups, CBI and counselling, including development, supervision and ongoing capacity building) costs 5.31 € (Burundi), 16.37 € (Indonesia), 6.66 € (Sri Lanka), 13.67 € (Sudan), per reached person (see Table 5). Comparisons between countries were difficult due to national economical differences, with GDP rates (in million of US Dollars) over 2008 varying from 1,163 for Burundi; 514,389 for Indonesia; 40,714 for Sri Lanka; 58,443 for Sudan—which in turn does not account for within-country economical differences (World Bank 2009). To demonstrate the costs of sustaining (as opposed to developing) a downsized intervention package, costs for development (i.e. capacity building) and focused care (i.e. counselling and supervision) were excluded, which resulted in cost per person of 3.46 € (Burundi), 8.47 € (Indonesia), 4.93 € (Sri Lanka), and 8.46 € (Sudan). The same analyses, but including only children who received services and excluding the number of parents and teachers reached through group psycho-education sessions, revealed that the cost per user increases to 5.37 € (Burundi), 15.63 € (Indonesia), 7.62 € (Sri Lanka), 10.78 € (Sudan), when considering running costs only. The above analyses across countries, revealed that mean cost per user ranges from 6.33 € (reduced cost and all beneficiaries), 9.85 € (reduced cost and screened children), 10.50 € (conservative cost and all beneficiaries), to 16.52 € (conservative cost and screened children).

Table 5.

Cost analyses

	Burundi	(USD)	Indonesia	(USD)	Sri Lanka	(USD)	Sudan	(USD)
Cost (conservative)a	181,169		171,334		125,334		135,954
Cost (reduced)b	117,982		88,594		92,739		84,119
Screened childrenc	21,970		5,670		12,174		7,805
All beneficiariesd	34,121		10,465		18,812		9,944
Mean cost per user (conservative cost & all beneficiaries)	5.31	7.06	16.37	21.77	6.66	8.85	13.67	18.18
Mean cost per user (reduced cost & all beneficiaries)	3.46	4.60	8.47	11.26	4.93	6.55	8.46	11.25
Mean cost per user (conservative cost & screened children)	8.25	10.97	30.22	40.19	10.30	13.69	17.32	23.03
Mean cost per user (reduced cost & screened children)	5.37	7.14	15.63	20.78	7.62	10.13	10.78	14.33

Note: Cost and number of targets are accumulated of entire project period (2005–2008). All costs are in Euros, except for the USD columns. USD rates were calculated by using an average conversion rate for the project period (average: 1 € = 1.33 $)

^aAll program costs included: project management, administrative and service providing staff, transport (including fuel, insurance and maintenance), capacity building of service providers, equipment, office costs, intervention and supervision costs (all interventions)

^bProgram costs excluding capacity building, counselling and supervision costs, other costs identical

^cScreened children includes beneficiaries receiving CBI, child resilience groups, counselling

^dAll beneficiaries includes screened as well as parents and teachers receiving group psycho-education

As can be seen in Table 5, the mean cost per service user ranged considerably between countries (ranging from 5.31 to 16.37 for the entire package, including development). By far the largest contributor to the cost was human resources, accounting for ~58 (Burundi), 59 (Indonesia), 56 (Sri Lanka) and 46% (Sudan). The next largest contributors to cost were transportation (9, 16, 3 and 18%, respectively) and capacity building (9, 12, 10 and 8%, respectively). Cost ratio was dependent on determination of beneficiaries, with reductions of mean cost per user ranging from 35% (Burundi), 46% (Indonesia), 36% (Sri Lanka) to 12% (Sudan) when including mental health promotion activities.

Discussion

While comprehensive multi-layered approaches are frequently advocated for psychosocial and mental health care in LAMIC (WHO 2008), few examples are presented in the literature, and hardly any evaluation is available (Jordans et al. 2009). An exception is a recent study by Layne and colleagues (2008) who conducted a simultaneous trial into the efficacy of first tier and second tier interventions of a 3-tier public health program in Bosnia. Another example is an evaluation by Wessells and Monteiro (2006) who used a combined quantitative and qualitative monitoring and evaluation system to demonstrate outcomes of a multi-component psychosocial care program in Angola. This paper aimed to present a practice-driven evaluation of a multi-component care package for children in four conflict-affected settings.

Results demonstrate that in low-resource settings with scarce available mental health care, development and implementation of a multi-layered care system is feasible and largely satisfactory, but is dependent on external financial support. Based on an adaptation of the filter model of Goldberg and Huxley (1980) we assessed the utilization of different levels of care. Relatively large percentages of children were screened for elevated psychosocial distress and subsequently much smaller percentages were referred to individualized or specialized care. The percentages of people indicated for the different levels of care are in line with population projections of distress and disorders in complex emergencies, i.e. 30–50% moderate chronic psychological distress, 10–20% mild to moderate common mental disorders and 2–4% severe mental disorders (Van Ommeren et al. 2005). The percentages for severe mental disorders are lower in our study, which is likely due to under-detection or to unavailability of services to refer to in two settings. With regard to treatment perceptions, we find high levels of client satisfaction for the group-based intervention and moderate-to-high levels for counselling across settings, and, albeit to a somewhat lesser degree, of perceived post-treatment problem reductions. In the literature, client satisfaction has commonly been associated with treatment gains (Holcomb et al. 1998; Veerman and van Yperen 2007). High levels of satisfaction are also found among therapists, yet this seems to come at some cost. Levels of therapist distress are significant. Client satisfaction is more prominent for the group intervention compared to counseling. We hypothesize this is due to the enjoyable, active and creative-expressive nature of the group intervention and the increased severity of problems seen by counselors. In contrast, therapist burden is higher for the group facilitators compared the counselors. While distress is high amongst both groups of service providers, the more thorough training of counselors and the increased status and remuneration are likely buffers for the perceived distress.

The presented data on CBI supplement efficacy research we conducted previously (Jordans et al. 2010; Tol et al. 2008) in several ways. The present study demonstrates high rates of perceived problem reductions and satisfaction, suggesting a high level of community acceptance of the intervention. Moreover, the beneficiary-mentioned indicators of post-treatment change are mostly social-behavioural (i.e. diligence, obedience, respect, social inclusion, self-esteem, shyness and interest in school), rather than symptoms of psychopathology. The trials present a more conservative perspective on treatment impact, likely due to scientific rigour of a trial and possible higher social desirable answering tendency in practice-driven evaluations.

Economic evaluation of mental health care in LAMIC is scarce, but has been shown feasible for community based mental health care in India and Pakistan (Chisholm et al. 2000). Cost analyses demonstrate that mean cost per service user varies heavily between countries and type of calculations, making between-country comparison difficult. While project expenditures were by no means excessive, the average cost (€10.50 per service user) is relatively high when compared to per capita health expenditures (Burundi $16; Sudan $54; Indonesia $118; Sri Lanka $163; UNDP 2008). Similarly, the cost per capita for the provision of a scaled-up core mental health care package (mainly pharmaceutical and psychosocial treatment for four mental health conditions contributing largely to the burden of disease) is between $1.85 and 2.60 for low-income countries (Chisholm et al. 2007; Lancet Mental Health Group 2007). At the same time, straightforward comparison with these figures is not possible as the presented costs in this paper are per reached service user. Costs should be aggregated to total coverage population numbers to get an actual per person per year figure, which was not feasible with the data available to us. Secondly, the presented costs reflect both development and sustainability of a basic psychosocial care package, as opposed to only care delivery costs as is common practice. Third, the following aspects may have had an impact on the average cost ratio: scaling up of the CBI intervention happened only after the termination of a phase focused on research; service delivery frameworks were developed without any existing infrastructure; the program focused on quality of service rather then on reaching the highest number of people. For example conducting clinical supervision and providing counselling increased costs significantly, while not increasing numbers in any considerable way. Nonetheless, the presented costs require external financial aid for most LAMIC and require more attention to establish cost-effective psychosocial and mental health care (Tomlinson et al. 2009). The cost analyses represent basic calculations. Presented data did not allow for more sophisticated cost-effectiveness or cost-benefit analyses of treatments.

The results have several implications. First, a combined primary and secondary screening procedure is feasible and facilitates detection and distribution of children over different available levels of care, thereby better matching limited resources with needs. Second, furthermore, high levels of distress among service providers need to be addressed for example by adapting workload, training, provision of care-for-caregivers and addressing logistic difficulties of service providers. Future qualitative assessments (e.g. focus group discussions) should elucidate the perceived causes of distress among the interventionists to inform the process of adaptations. Third, we argue that evaluations of care packages should ideally combine efficacy and cost-effectiveness research into separate treatments within the package with evaluations that emphasize practice driven evaluations (i.e. recipient perspectives, cost; Veerman and van Yperen 2007). A combined approach permits triangulation of trial outcomes with evaluations focusing on beneficiaries’ perspectives, cost analyses or other indicators of evaluation. Fourth, there is an obvious need for further cost reductions to increase sustainability. More children need to be reached with the same human resources (cost) to make a care package such as the one presented financially feasible.

Conclusion

This study evaluated a multi-layered psychosocial and mental health care delivery framework for children in four conflict-affected countries. It demonstrated access to, and utilization of, different levels of care within resource-poor settings. Practice-driven evaluations demonstrated high levels of satisfaction combined with considerable levels of perceived post-treatment problem reductions and treatment gains. Therapist burden is higher than expected, especially for community workers or for lower perceived client progress. Analyses of cost per service user give rarely reported data on actual cost of development and implementation of community based psychosocial care in LAMIC.

Appendix

See Fig. 2.

Fig. 2.

Four project countries