Process evaluation improves delivery of a nutrition‐sensitive agriculture programme in Burkina Faso

Abstract

Evidence is emerging from rigorous evaluations about the effectiveness of nutrition‐sensitive agriculture programmes in improving nutritional outcomes. Additional evidence can elucidate how different programme components and pathways contribute and can be optimized for impact. The International Food Policy Research Institute, with Helen Keller International, designed a comprehensive framework to evaluate the delivery, utilization, and impact of Helen Keller International's enhanced homestead food production programme in Burkina Faso. After 18 months of implementation, a process evaluation was conducted to examine programme impact pathways, using key informant and semistructured interviews with implementing agents and beneficiaries, and with residents of control communities. Data were analyzed by International Food Policy Research Institute and reviewed with project managers and partners through multiple workshops to identify opportunities to strengthen implementation. Findings illuminated gaps between intended and actual delivery schemes, including input constraints, knowledge gaps among community agents in agriculture and young child nutrition practices, and lower than expected activity by community volunteers. In response, staff developed measures to overcome water constraints and expand vegetable and poultry production, retrained volunteers in certain techniques of food production and counselling for nutrition behaviour change, added small incentives to motivate volunteers, and shaped both immediate and long‐term changes to the programme model. Working closely with International Food Policy Research Institute on the evaluation activities also expanded the repertoire of research methods and skills of Helen Keller International staff. Process evaluation can strengthen programme delivery, utilization, and design. Collaboration between researchers and implementers can improve programme effectiveness, project staff capacity, and advance delivery science.

 

Key messages.

• Complementing a cluster‐randomized impact evaluation with a process evaluation that examines actual implementation and utilization allows for mid‐course correction, better interpretation of impact results, and more effective programmes.

• Close collaboration among key partners (implementing, research and government) led to the successful and timely implementation of several mid‐course corrections that may have contributed to better impacts.

• Results from the process evaluation and impact evaluation taken together informed improvements in the programme model applied in Burkina Faso as well as those implemented in different settings.

• Collaboration on this kind of rigorous implementation research also strengthened the professional capacity of the implementing team.

1. INTRODUCTION

In the years since the 2008 Lancet Series on Maternal and Child Undernutrition, there has been a groundswell of interest in scaling up proven interventions that address both the immediate and underlying causes of undernutrition in women and children (Shekar et al., 2016). Although a growing evidence base demonstrates the effectiveness of nutrition‐specific programme approaches for improving nutrition outcomes, further research is needed to understand how such programmes achieve impact (Garrett, 2008), to ensure programme delivery aligns with programme design (Black et al., 2013) and to inform mid‐course corrections to strengthen delivery. In addition, more evidence is needed on the impact of nutrition‐sensitive interventions on nutrition outcomes and on the pathways to impact (Ruel & Alderman, 2013). Although randomized control trials are considered the gold standard for providing evidence on the efficacy or effectiveness of interventions (Habicht, Victora, & Vaughan, 1999), fully understanding this impact in the context of complex interventions requires research to examine the degree to which the programme was implemented and utilized as planned and to confirm the validity of the programme theory on which the programme is based or to inform revisions (Habicht & Pelto, 2014). To date, most reviews have assessed the existing evidence base for nutritional outcomes (Berti, Krasevec, & FitzGerald, 2004; Girard, Self, McAuliffe, & Olude, 2012; Masset, Haddad, Cornelius, & Isaza‐Castro, 2012) without considering how successfully the programmes were delivered.

Increasingly, programme planners are articulating programme impact pathways (PIP) to describe how they expect the inputs invested and activities designed to lead to the intended outputs, outcomes, and ultimate impact. The schema should identify all the steps and components required to achieve impact, as well as indicators that can be used to assess progress and programme performance. This theory should then inform the design of a process evaluation (PE) that collects data along the delivery chain to assess how well activities are being implemented and utilized, (insert comma ‐‐ unless that is US grammar) and the extent to which objectives are being achieved. For example, interviews with project staff and beneficiaries can assess beneficiary participation in training, acquired knowledge, improved and applied skills in areas covered (e.g., gardening and nutrition counselling), and whether counselling is motivating clients to adopt desired behaviours. The usefulness of such findings can be increased if compared with data from communities not receiving the training and related inputs. PE can also deepen understanding of potential barriers and facilitators of programme participation and uptake, identify components that are not working as expected, and inform adjustments to activities. For example, if training participants cannot recall key concepts, the training content or methodology can be revisited. PE can be used to verify whether the expected “dose” for key interventions was delivered, which can contribute to the plausibility of any programme impacts achieved (Victora, Habicht, & Bryce, 2004). Thus, if community health workers are faithful in organizing monthly nutrition discussions with mothers and in making the expected number of home visits to support families to adopt new practices, it is more likely that measured behaviour change can be attributed to those activities. Together, impact and process evaluation can confirm that the programme theory is valid (the programme was delivered according to the theorized pathways and outcomes and impact were achieved) or guide revisions (if the programme was delivered as intended, but the intended impact was not achieved).

From 2009 to 2012 with funding from the U.S. Agency for International Development, Helen Keller International (HKI) partnered with the International Food Policy Research Institute (IFPRI) to apply this layered evaluation approach to its enhanced homestead food production (EHFP) programme in Eastern Burkina Faso. The findings of the community randomized impact evaluation have been reported elsewhere (Olney et al., 2016; Olney, Pedehombga, Ruel, & Dillon, 2015) and include significant reductions in the prevalence of anaemia among infants 3–6 months of age at baseline, wasting among children 3–12 months at baseline (Olney et al., 2015), and underweight among mothers (Olney et al., 2016; Clinical Trial NCT01825226). Here, we report on some of the key findings from the PE conducted 1 year after programme implementation began (Olney, Behrman, Iruhiriye, van den Bold, & Pedehombga, 2013), how the findings were used to refine programme implementation and rectify weaknesses in real time through a series of consultations. Another study from this programme examined the challenges of integrating interventions across multiple sectors and outlined lessons learned (Harris & Buchsbaum, 2014).

2. METHODS

2.1. Study area and programme design

Food insecurity and child undernutrition are important problems in Burkina Faso. At the time of project design, the prevalence of wasting (height for weight z‐scores <−2SD reference norms) and stunting (height for age z‐scores <−2SD reference norms) in the Eastern region were estimated at 17.7% and 42.8%, respectively, and anaemia prevalence (Hb < 11.0 g/dl) was 91.2% in children 6–59 months and 53.1% in women of reproductive age (Institut National de la Statistique et de la Démographie—INSD/Burkina Faso & ICF International, 2012). The population remains extremely vulnerable, with a ranking of “serious” on the Global Hunger Index (von Grebmer, Bernstein, Prasai, Yin, & Yohannes, 2015) and 44.5% living under the poverty line of $1.25 per day (United Nations Development Programme, 2015).

To address these challenges, HKI adapted its well‐known EHFP model from the Asian context, where it was developed and has been shown to improve multiple nutrition indicators (Haselow, Stormer, & Pries, 2016), to the semi‐arid context of the African Sahel. In Burkina Faso, the model used community gardens as platform to train and empower women producers, including volunteer village farm leaders (VFL) and the women's group members they mentor, to increase the diversity and volume of nutrient‐dense foods produced and consumed by households. The women also met monthly with community health volunteers trained to use techniques of interpersonal and group behaviour change to promote the adoption of evidence‐based optimal nutrition practices (Bhutta et al., 2013). These activities were targeted to the beneficiary mothers and focused on practices related to health, nutrition, and hygiene important during the critical period between the beginning of pregnancy to a child's second birthday. Details of the implementation strategy can be found elsewhere (Olney et al., 2015).

Throughout programme design and implementation, HKI worked in close partnership with the government at multiple levels. A project steering committee chaired by the Provincial High Commissioner reviewed the work plan, research findings, and lessons learned and provided technical guidance on implementation. The committee met biannually with participants from the departments of health, agriculture, livestock, environment and women's affairs, local elected and administrative officials, and non‐governmental partners working in the province. A local non‐governmental organization, Association d'Appui et de Promotion Rurale du Gulmu (APRG), served as HKI's implementing partner, providing training and supportive supervision together with staff of district health and agriculture offices to community volunteers.

Horticulture and poultry production were established first at each community garden, where four VFL were trained by master trainers in improved techniques for diversified plant production and poultry breeding. VFL, supported by master trainers, then trained between 20 and 40 women per village organized as mothers' groups. After the first growing season, mothers' group members received support to develop home gardens, whereas the VFL continued to share responsibility for, and harvest, the community garden, and to serve as technical resources.

Monthly nutrition discussions for mothers' groups were facilitated by teams of community health volunteers trained in the Essential Nutrition Actions (ENA) framework (Guyon et al., 2009). Project supervisors from APRG received a 5‐day training with more technical detail; a simplified 3‐day version was used with volunteers. These teams were composed of either the designated community health committee (HC) or a group of older women leaders (OWL). Volunteers were taught to apply techniques designed to motivate families to adopt recommended practices during meetings and through home visits.

The project began in July 2009 with staff recruitment, followed by planning and design of both the research and implementation strategies. Delivery of programme interventions began after the completion of the quantitative baseline survey (February–May 2010) and was assessed exactly 2 years later by the same instrument.

2.2. Evaluation design

IFPRI designed all research studies and instruments and led the data collection and data analysis activities in close collaboration with HKI field and headquarters staff. The full description of the impact evaluation has been reported elsewhere (Olney et al., 2016; Olney et al., 2015). Briefly, IFPRI developed a sampling frame covering four departments of the Gourma province and randomly allocated 55 villages to one of three study arms: (a) treatment group receiving the EHFP programme with the nutrition component delivered by a six‐member HC (15 villages); (b) treatment group receiving EHFP with the nutrition component delivered by a team of six OWL (15 villages), and (c) control villages (n = 25) receiving no intervention. In addition to measuring maternal and child nutrition outcomes, the design sought to assess whether there were differences in the effectiveness of nutrition counselling delivered by the HC versus OWL.

The PE was conducted midterm, in April and May 2011, using mixed methods. The objectives were fourfold: (a) investigate if implementation and utilization were proceeding as designed; (b) evaluate the quality of programme activities; (c) invite feedback for improvements from beneficiaries and implementers; and (d) identify ways to strengthen delivery and/or utilization of the programme components to increase ultimate impact. The design and analysis of the PE were guided by the PIP model adapted from a version jointly designed by IFPRI, HKI, and local stakeholders for HKI's programme in Cambodia (Olney, Richter, et al., 2013; Figure 1). The model distinguishes aspects of programme delivery and utilization and highlights three pathways to impact, each of which required contributions from the government as well as APRG and HKI:

• Improved production: VFL are trained and equipped to apply more productive cultivation techniques and to transfer these skills to beneficiary women producing at the household level. Project training, inputs, and supportive supervision lead to increased variety and volume of nutrient‐rich plant‐ and animal‐source foods produced. Increased production, in turn, contributes to improved household food and nutrition security, improved dietary intake, and, ultimately, improved nutritional status.

• Increased empowerment and income: Women gain assets and skills that can contribute to empowerment and income generation, which can be used to acquire higher quality foods and health care, and thus improved nutritional status.

• Improved knowledge and practices: OWL and HC are trained in the ENA curriculum, master the contents, and apply negotiation for behaviour change techniques in interpersonal and group discussions to encourage families to adopt optimal nutrition, health and hygiene practices, which can improve nutritional status.

Figure 1.

Programme impact pathways schema

Semistructured interviews (SSI) containing both open‐ended and precoded questions were conducted with randomly selected beneficiaries from five randomly selected households within each intervention village (total treatment SSI = 145) and from 15 villages randomly selected from the 30 control villages (total control SSI = 75). Control communities were included to provide a comparison group. Two SSI respondents in each intervention village were randomly selected to answer additional questions probing certain topics. Key informant interviews (KII) were conducted with a purposive sample of project master trainers in agriculture (n = 13) and nutrition (n = 24), as well as with two VFL (n = 58) and two nutrition volunteers (n = 30 OWL; n = 28 HC) from each intervention village. Table 1 presents the interviews completed by respondent group and village. The methodology is detailed in IFPRI's process evaluation report (Olney, Behrman, Iruhiriye, van den Bold, & Pedehombga, 2013).

Table 1.

Sample sizes and subjects for semistructured and key informant interviews

	Beneficiary villages		Control villages	Ag master trainers	Nutrition master trainers
	OWL villages	HC villages	Control villages
SSI with mothers	75 (5)	70 (5)	75 (5)
KII				13	24
Village Farm Leaders	58 (2)
OWL	30 (2)
HC		28 (2)

The numbers in the table are the total number of assessments completed for each group, and the number in parentheses is the number completed per village for each group. SSI = semistructured interview; KII = key informant interview; OWL = older woman leader; HC = health committee.

The interviews probed informants' level of participation in the programme, opinions of the quality of training and inputs provided, knowledge and practices in the project domains (nutrition, gardening, and animal husbandry), perceptions of household food security, dietary diversity, participation by women in household decision‐making, labour and time demands of project activities, and income generated from production. Master trainers and community agents were also asked about their training methodology, perceptions of which topics they and their beneficiaries found easier and harder to understand, level and sources of motivation, opinions about benefits to the community, and recommendations for changes to increase programme benefits. Agricultural trainers and volunteers were asked about constraints to production, whereas nutrition volunteers were asked to identify the practices easiest and hardest to change and factors that support or discourage change. Quantitative data were assessed with descriptive analysis using SPSS. Components with positive/correct responses of >75% were coded as working well; 25–75% as needs improvement; and <25% as not working. This classification has been used in other process evaluations to estimate programme performance and identify components most in need of improvement (Olney, Parker, Iruhiriye, Leroy, & Ruel, 2013; Olney, Richter, et al., 2013; Olney, Vicheka, et al., 2013).Qualitative data were transcribed and analyzed manually and coded for key themes guided by the PIP (Olney, Behrman, et al., 2013).

A workshop was held in November 2011 to review the findings and recommendations with implementers and key stakeholders from the government and to discuss priority actions for strengthening delivery and utilization. Between January and March 2012, a series of similar meetings were held with the project steering committee and other technical partners. The four‐member implementing team then continued discussions internally to identify additional solutions to problems revealed by the research. These efforts led to adjustments detailed below.

2.3. Ethical approval

Ethical approval for all studies was provided by the Ministry of Health of Burkina Faso and by IFPRI's institutional review board.

3. RESULTS

Table 2 presents selected findings from the SSI with mothers from intervention and control communities and KII implementing agents.

Table 2.

Selected quantitative findings from beneficiary and nonbeneficiary interviews, n (%)

Indicator	Beneficiary responses	Nonbeneficiary responses
Production knowledge, attitudes, and practices
VFL understood nursery preparation	37/58 (64)	N/A
Mothers understood nursery preparation	48/58 (83)	N/A
VFL understood training in plant care	46/58 (79)	N/A
Mothers understood training in plant care	37/58 (64)	N/A
Had a home garden	114/136 (84)	3/74 (4)
Water shortage perceived as production constraint	85/114 (83)	3/3 (100)
Earned income from vegetable sales	22/54 (41)	0
Earned income from egg sales	22/53 (42)	14/27 (52)
Woman keeps income from vegetable sales	35/43 (82)	0
Felt vegetable production improved through programme support	37/44 (84)	N/A
Satisfied with level of vegetable production	23/54 (43)	N/A
Satisfied with level of poultry production	29/55 (53)	N/A
Perceived programme improved food security	52/55 (95)	N/A
Nutrition knowledge
OWL understood proper positioning for breastfeeding	25/29 (86)	N/A
HC members understood proper positioning for breastfeeding	21/28 (75)	N/A
OWL understood complimentary feeding	27/29 (93)	N/A
HC understood complimentary feeding	24/28 (86)	N/A
OWL understood topic of feeding the sick child	8/29 (28)	N/A
HC understood topic of feeding the sick child	12/28 (43)	N/A
OWL visited mother to check on adoption of practices	10/29 (35)	N/A
HC visited mother to check on adoption of practices	19/28 (68)	N/A
OWL could cite two ways to prevent anaemia	9/30 (30)	N/A
HC could cite two ways to prevent anaemia	12/28 (43)	N/A
Mothers could cite two ways to prevent anaemia	39/136 (29)	17/70 (24)
Mothers know two foods to enrich porridge	119/136 (86)	50/70 (71)
Mothers know two foods rich in Vitamin A	107/130 (82)	34/56 (61)
Mothers trusted volunteers' advice (OWL communities)	20/29 (69)	N/A
Mothers trusted volunteers' advice (HC communities)	24/28 (86)	N/A

Note. OWL = older woman leader; HC = health committee; VFL = Village Farm Leaders; N/A = not applicable.

3.1. Production and income pathways

Because improving production was the vehicle for increasing income, these two pathways are discussed together. Overall, the findings suggested improvements were needed to the HFP training at both VFL and household levels and increased technical and material support were needed to strengthen horticultural and animal production. Water shortages were identified by the great majority (>90%) of respondents as a serious constraint. Although in defining the sampling frame the project team sought to assure water resources were adequate (Behrman, Dillon, Moreira, Olney, & Pedehombga, 2011), data were quite limited. As production expanded, challenges emerged: existing boreholes were often insufficient to meet the additional requirements, and pumps broke down from heavy use.

HKI took this finding seriously, as production depended on adequate water. Several solutions were pursued, some suggested by beneficiaries during interviews. At community gardens, these included constructing four new gardening wells and two boreholes; repairing 12 existing boreholes in partnership with village water management committees; installing five pedal pumps; supporting construction of one low‐pressure pipe irrigation system at a garden located by a dam; and relocation of two gardens to sites with greater proximity to water. Techniques to reduce water use were promoted in subsequent trainings: mulching, prioritizing plants with short growing cycles and those requiring less water, and using the rainy as well as dry season to grow nutrient‐rich crops. A simple technology drip irrigation system was introduced in community and household gardens, but unfortunately, proved impractical due to the high silt concentration in the water and eventually was abandoned.

An important proportion of VFL (23/58; 40%) and mothers (14/58; 26%) reported that their estimated gardening workload of 3–4 hr per day interfered with other responsibilities. This finding was of concern given women's already high work burden. To address this, husbands were invited to participate in refresher training and to contribute their labour to this family enterprise. Health volunteers were encouraged to include fathers in the monthly nutrition discussions, demonstrate to them the benefits to the household of optimal child feeding, and motivate their support in this responsibility. Scripts for community theatre and radio broadcasts reiterated these themes. The project also intended to enhance women's control over agricultural assets. Although almost all beneficiaries reported having authority over gardening activities, just over half reported controlling revenue from poultry.

Interviews with master trainers and VFL indicated that the horticulture training was appreciated as high quality and practical. Most beneficiaries (103/114; 90%) reported that their gardens, and the production techniques applied, were the result of project support. Although almost all VFL respondents claimed to understand the technical topics (e.g., nursery preparation, soil fertilization, pest control, and water conservation), one third of master trainers felt VFL might not have sufficient mastery of some topics to teach beneficiaries. To reinforce knowledge, refresher trainings were organized for all VFL and direct beneficiaries. The opportunity was also used to invite interested members of nonbeneficiary households within the intervention communities to participate as observers, per their request. Field supervision was intensified to ensure techniques were correctly applied at community and home gardens.

The PE findings provided richer details to the project team about the challenges observed in their daily routines. Although almost all beneficiaries stated they believed the programme had improved their techniques, only half reported being satisfied with the level of poultry (29/55; 53%) or vegetable (23/54; 43%) production achieved, citing challenges of high mortality, insufficient water, and pests. They requested increased support for poultry vaccination, boreholes, fencing, gardening tools, and access to more seed varieties. For poultry, the project team introduced an improved poultry coop design, feed recipes with increased mineral and protein content using affordable local ingredients, and brooding techniques that combined improved and local breeds to favour both reproduction and disease resistance. HKI also negotiated an agreement with the provincial department of animal husbandry to reinforce vaccine delivery. For gardening, the project distributed additional tools, seeds, and supplies and provided training on suitable fence construction to keep out foraging animals.

About one third of beneficiaries interviewed believed that their sales from plant and animal production had increased through project support, and almost half reported having sold surplus production. However, most beneficiaries reported that food produced was primarily used to improve household consumption, per recommendations in the nutrition discussions. Most beneficiaries (52/55; 95%) perceived the programme had improved their food situation, highlighting increased consumption of vegetables (49/55; 89%) and eggs (28/55; 51%). Household dietary diversity scores based on a 24‐hr recall of 17 food categories were slightly higher among beneficiaries than nonbeneficiaries (4.1 vs. 3.8) with the largest difference being in “other fruits and vegetables.” Recognizing the significant production constraints, the project team emphasized household consumption rather than sales, augmenting dry season gardening with additional vegetable and pulse production during the main cultivation season, which normally centers on staple crops.

3.2. Knowledge and practice pathway

Across the board, those trained in the nutrition curriculum felt the training was high quality and valuable. Most master trainers interviewed felt well prepared to train village volunteers (23/24; 96%), believed they were given clear guidance from HKI on how to conduct this training (24/24; 100%), felt motivated by the opportunity to help reduce child malnutrition (17/23; 74%), and believed the community volunteers appreciated their work (17/24; 71%). However, many expressed concerns that OWL's low education levels might limit their mastery of the subject matter and effectiveness in communicating ideas accurately to beneficiaries. They recommended additional review or refresher trainings, as did most OWL volunteers themselves (19/29; 66%). About one third of volunteers (OWL 5/29, 17%; HC 9/28, 32%) indicated they felt the burden of travel and opportunity costs were high and warranted compensation.

Although master trainers generally felt they understood most topics well, two appeared difficult: feeding the sick child (7/25; 29%) and enriching complementary foods (2/24; 8%). Community volunteers also cited difficulties understanding feeding the sick child (3/57; 5%) and preventing anaemia (only 21/58 [36%] could cite two ways). HC expressed more confidence in their overall understanding of topics (21.28; 75%) than did OWL (14/29; 48%), and were less likely to ask for additional training (10/28; 36%) than were OWL (19/29; 66%). OWL were less likely than HC to be able to identify one way to prevent malaria and two ways to prevent anaemia.

In the KII, volunteers reported that they discussed ENA topics with women both individually and in groups and that they followed up to ensure women understood the concepts (45/57; 79%); beneficiaries largely reported these discussions to be of good quality and useful (52/55; 93%). Volunteers noted that although most of those reached were direct beneficiaries, about three quarters of their group discussions included some nonbeneficiary women and about one quarter included men from their communities.

The KII revealed that although most community volunteers used persuasion to encourage mothers to adopt recommended practices (42/57; 74%), a small proportion (7/57; 12%) reported reprimanding mothers for not following their advice, despite the training's emphasis on negotiating behaviour change. Moreover, only about half of the community volunteers reported checking with mothers as to whether they had adopted the promoted practices (29/59; 51%). HC were more likely to do so (19/28; 68%) than OWL (10/29; 35%). HC (5/28; 18%) were also more likely than OWL (0/29; 0%) to recruit the support of additional family members if needed. Although the majority of volunteers interviewed reported making home visits, only 45% (60/133) of the beneficiaries interviewed had received the expected two home visits, whereas 39% (52/133) had not received any in the past month. Moreover, among beneficiaries who had not received a home visit in the past month, a majority had never received a home visit (38/52; 73%). Many expressed a wish for more home support (22/55; 40%) or to have concepts explained more carefully (17/55; 31%). All these strategies are important aspects of negotiating for behaviour change.

Although greater proportions of beneficiaries could answer knowledge questions on infant and young child feeding correctly compared to controls, there were few differences in actual feeding practices between the intervention and control groups. An important exception was that a higher proportion of beneficiary children (37%; 50/136) were fed eggs in the previous 24 hr than were nonbeneficiary children (16%; 11/70). About half of the beneficiaries reported facing barriers to adopting recommended practices (27/55; 49%), the most important being money; smaller proportions cited the limited availability of recommended foods, difficulty understanding the recommendations, lack of time, or respect for tradition. Notably, beneficiaries in the OWL villages were less likely than HC beneficiaries to express trust of the community volunteer's teaching that the new practices were better and cited this lack of trust as a barrier to adopting new practices (OWL = 9/29; 31% vs. HC = 4/28; 14%). Beneficiaries in HC villages were also more likely than OWL beneficiaries to report receiving support from their husbands (HC = 21/28; 75% vs. OWL = 16/27; 55%) or other community members (HC = 12/28; 43% and OWL = 5/27; 17%) to adopt new practices, presumably due to the efforts of the HC volunteers.

The implementing team concluded that all cadres would benefit from refresher training to review key topics, notably the causes and symptoms of anaemia, identifying iron‐rich foods, the prevention of malaria, feeding the sick and recuperating child, importance of eggs for young children, women's nutrition, and fathers' role in supporting family nutrition. A series of radio spots and call‐in contests were developed to reinforce understanding of the prevention of anaemia and feeding the child 6–24 months.

Additional training was also provided to master trainers and volunteers to strengthen skills for negotiating for behaviour change, facilitating participatory learning, and seeking solutions to barriers to adoption of best practices. New job aids included illustrations showing the small amounts of food consumed by children 6–24 months of age to help persuade families the cost could be manageable. The gardening and poultry components of the strategy were expected over time to reduce the obstacles of cost and access by increasing the volume of home production of nutrient‐dense plant and animal foods. To increase consumption of iron‐rich foods, the project team promoted more legumes and dark green leafy vegetables in gardens and consumption of widely available wild sources such as baobab and moringa leaves and locust beans. The intercropping of new vegetables with rainy season crops expanded access during the period when families were required to leave behind home gardens to cultivate the staple crops.

The project team weighed the issues of volunteer motivation and chose to honour contributions through public ceremonies and small practical gifts (e.g., soap, watering cans, seeds, and T‐shirts), to reinforce the value of service for community betterment. In some cases, volunteers were provided income‐generating opportunities as orange‐fleshed sweetpotato (OFSP) vine suppliers. Adapting meeting times to local preferences and adding cooking demonstrations increased participation and further encouraged volunteers.

3.3. Capacity building

A concomitant benefit of the partnership between the local implementation team and the IFPRI was the expansion of implementers' skills and appreciation for rigorous research design, data quality assurance, and analysis. Supporting IFPRI experts in developing the sampling frame and data collection instruments, planning field work, training enumerators and reviewing findings exposed the project team to best practices in impact and process evaluation. The project coordinator and M&E coordinator also participated in international workshops that introduced them to other research and implementation teams to exchange learnings and methodologies. Although the challenges of managing the competing demands of multiple rounds of intensive data collection with those of a complex, multisectoral intervention programme were considerable, they were compensated by the returns from mid‐course corrections and a more effective programme model, increased capacity of HKI staff, and contributions to the global evidence base for effective public health and development investments.

4. DISCUSSION

The PE found that many elements of the intervention were being implemented as intended. Nevertheless, important weaknesses were identified and informed adjustments to the implementation strategy. These findings, together with those from the impact evaluation, also shaped the design a second phase of the project that began in 2013 with support from the government of Canada. Modifications included further attention to building volunteers' skills in counselling for behaviour change; stronger supportive supervision systems; additional interventions aimed at reducing anaemia due to both nutritional deficiencies and infectious causes; new strategies to address the presumed contributions of inadequate water, hygiene, and sanitation conditions to malnutrition; and a more consistent motivation scheme for community volunteers. HKI also introduced more frequent, rapid monitoring surveys to complement the in‐depth PE with regular assessments of activity roll‐out.

Our study contributes to a growing body of implementation research using varied and rigorous methods to assess how well nutrition programmes are delivered and utilized compared to plans (Avula et al., 2013; Kim et al., 2015; M. N. N. Mbuya et al., 2015; Menon, Rawat, & Ruel, 2013; Rajaraman et al., 2014), how to optimize programme implementation (Olney, Vicheka, et al., 2013; Winch et al., 2008) and to provide confirmatory evidence of the posited causal pathways (Loechl et al., 2009; Robert et al., 2006). Like ours, other studies have identified breakdowns in the pathway between training and effective application of its contents, including the volume of content to be absorbed, the skills and attributes of trainers, logistical and human challenges to adequate supervision, and the considerable and perhaps unrealistic demands placed on community agents (Kim et al., 2015; Mbuya, Menon, Habicht, Pelto, & Ruel, 2013). Such research has documented exposure to the intervention (Avula et al., 2013; Rajaraman et al., 2014) as well as gaps in delivery (Mbuya et al., 2013) and has deepened understanding of the complexities of such multisectoral public health undertakings (Cole et al., 2016; Harris et al., 2016). In our study, although master trainers evinced good understanding of the ENA concepts, there was considerable loss through the cascade to OWL/HC and then mothers, which led us to increase the frequency of refresher training and supervision and reinforce communication skills. Such research also reveals the significant challenges of translating even carefully designed programmes into effective behaviour change activities on the ground in low‐resource settings (Avula et al., 2013; Cole et al., 2016; Harris et al., 2016; Kim et al., 2015). Further losses occur in the translation of knowledge into practice, for reasons that are complex and multifaceted (Aunger & Curtis, 2016). Yet research such as this should help enhance understanding of the impediments and the long‐term nature of behaviour change.

Our study had some limitations. The short length of the project and its fixed resources limited the scope for addressing major weaknesses identified. On the other hand, HKI has been implementing EHFP globally for decades and has been able to apply learnings and new evidence across settings; for example, incorporating animal husbandry to enhance dietary quality and nutrition education to promote consumption (de Pee & Bloem, 2007), replacing nutrition education with behaviour change communications and strengthening approaches to advance gender equity (Haselow et al., 2016). The collaboration with IFPRI has made important additional contributions to this process, including PE examining the delivery of an EHFP programme in Cambodia (Olney, Vicheka, et al., 2013), which also informed both immediate steps for strengthening implementation and improvements to programme design.

Many of the challenges identified were beyond the means of the project to surmount, including water constraints and unfavourable climactic conditions for homestead food production. Major public investments in agricultural research and extension, as well as in infrastructure, input and supply systems, and markets and by the government and the global donors and investors will be required to achieve the needed improvements in food security and dietary diversity (U.S. Department of State, 2016). The low educational level of our community nutrition volunteers may have hampered their ability to master the considerable technical content and related skills of the curriculum such as the problem‐solving techniques of negotiating for behaviour change. Another enduring challenge is incentivizing community health workers by, inter alia, mobilizing public sector resources to compensate them or selecting individuals with intrinsic motivation (Haines et al., 2007). Evidence suggests that these cadres can make significant contributions to public health, yet must be integrated fully into the health system, given adequate training, supervision, logistical support, and career growth opportunities (Perry, Zulliger, & Rogers, 2014). Burkina Faso is making strides in this direction, including reflection on appropriate incentive mechanisms (Rasanathan et al., 2014), which could eventually improve the calibre and performance of groups like our OWL and HC workers.

5. CONCLUSIONS

It has been argued that the majority of child deaths could be prevented simply by improving the delivery of proven interventions (Leroy, Habicht, Pelto, & Bertozzi, 2007). To do so, public health programmes need PE that reveals factors that undercut and buttress implementation. Such evidence will inform stronger delivery models and enhance the plausibility of ultimate outcomes (Habicht & Pelto, 2014). Our research contributed evidence at all these levels, allowing the implementation team to improve programme implementation in real time to rectify gaps in participant learning, production potential, and supervision structures. It also contributed to improving HKI's overall programme model and suggested additional changes to potentially increase programme impacts, some of which were tested in a follow‐on study. Investing in such research is essential to advancing understanding of how programmes contribute to reducing malnutrition and of the larger systemic issues that must be addressed to facilitate the implementation of complex programmes.

CONFLICTS OF INTEREST

The authors declare that they have no conflicts of interest.

CONTRIBUTIONS

JNN led the drafting of the manuscript based on interviews with MO, AP, and HR and provided technical input to the design and implementation of the project. DKO designed the evaluation with input from MO, AP, and HR, led data collection and analysis, and made extensive contributions to the manuscript. MO, AP, and HR led project implementation including the integrating of evaluation findings into revised delivery strategies. FYW advised on implementation, consultation with partners, and interpretation of evaluation findings for strengthening delivery. All authors read and approved the final version of the paper.

Nielsen JN, Olney DK, Ouedraogo M, Pedehombga A, Rouamba H, Yago‐Wienne F. Process evaluation improves delivery of a nutrition‐sensitive agriculture programme in Burkina Faso. Matern Child Nutr. 2018;14:e12573 10.1111/mcn.12573