Theory-driven process evaluation of a complementary feeding trial in four countries

Abstract

We conducted a theory-driven process evaluation of a cluster randomized controlled trial comparing two types of complementary feeding (meat versus fortified cereal) on infant growth in Guatemala, Pakistan, Zambia and the Democratic Republic of Congo. We examined process evaluation indicators for the entire study cohort (N = 1236) using chi-square tests to examine differences between treatment groups. We administered exit interviews to 219 caregivers and 45 intervention staff to explore why caregivers may or may not have performed suggested infant feeding behaviors. Multivariate regression analysis was used to determine the relationship between caregiver scores and infant linear growth velocity. As message recall increased, irrespective of treatment group, linear growth velocity increased when controlling for other factors (P < 0.05), emphasizing the importance of study messages. Our detailed process evaluation revealed few differences between treatment groups, giving us confidence that the main trial’s lack of effect to reverse the progression of stunting cannot be explained by differences between groups or inconsistencies in protocol implementation. These findings add to an emerging body of literature suggesting limited impact on stunting of interventions initiated during the period of complementary feeding in impoverished environments. The early onset and steady progression support the provision of earlier and comprehensive interventions.

Introduction

More than 10 million preventable deaths occur before the age of 5 years; malnutrition is a contributing factor to many of these deaths [1]. Exclusive breastfeeding and good complementary feeding have been cited as critical preventative measures to reduce excess mortality among children <5 years of age [2]. Exclusive breastfeeding during the first 6 months after birth provides important survival and nutritional benefits to young infants. As the infant approaches 6 months of age, complementary foods must meet the nutrient gaps that develop because of the longitudinal changes in milk composition and the older infant’s nutritional requirements [3–5]. Promotion of optimal complementary feeding, especially in conjunction with infection control [2], has also been identified as an effective intervention to reduce stunting and its associated adverse outcomes [6]. The World Health Organization (WHO) recommends, if possible, the daily consumption of meat, poultry, fish and eggs [7], which have a high-energy density, high-quality protein and highly bioavailable micronutrients, including iron and zinc. Due to its nature, the promotion of improved complementary feeding among mothers in low socio-economic populations is complex and requires different foods for different cultures. Interventions to promote dietary diversification with locally available foods have been undertaken, and some have shown promising results on growth [5].

The ‘Eunice Kennedy Shriver’ National Institute of Child Health and Human Development (NICHD) Global Network for Women’s and Children’s Health Research First Bites Complementary Feeding Trial was carried out to test the hypothesis that daily intake of 30–45 g of meat from 6 to 18 months of age would result in greater linear growth velocity and improved micronutrient status in comparison with an equicaloric multi-micronutrient fortified cereal. The details of the main trial protocol have been published elsewhere [8]. The primary outcome for the main trial did not differ between the two feeding groups, and both demonstrated progressive linear growth faltering. Maternal education level was the most strongly significant predictor of linear growth velocity, independent of the intervention group [9]. We conducted a theory-driven process evaluation of this cluster randomized controlled trial (RCT) to explore factors that may have facilitated or impeded the trial as well as factors that may have impacted the trial outcomes.

Methods

The trial was conducted in rural communities in the Democratic Republic of Congo and Zambia, semi-rural communities in the Western Highlands of Guatemala and urban communities in Karachi, Pakistan. Daily portions of study food and educational messages to enhance complementary feeding were delivered to caregivers individually by study coordinators, hereafter referred to as community coordinators. The three main educational messages delivered during home visits by community coordinators to caregivers were to: (i) provide a thick puree/gruel, (ii) feed complementary foods at least three times per day and (iii) maximize dietary diversity. As per the protocol, the frequency of home visits decreased during the course of the study from daily to three times per week to weekly. The protocol was approved by ethics boards located in the countries where the studies were conducted, the partnering US-based institutions and the Data Coordinating Center at RTI International.

We examined data collected during the trial to address the following process evaluation indicators: reach, dose delivered, dose received, fidelity and context as further described in Table I [10].

Table I.

Summary of process evaluation indicators

Process evaluation indicator	Definition	Measurement description	Data source
Reach	Proportion of the target audience that participated in the study	Early termination from the study by treatment group	Of entire study cohort (N = 1236), 174 participants did not complete the study
Dose delivered	Proportion of study components that was delivered to participants. Measurement focus is study staff	Study staff completion of home visits and study food distribution by treatment group	Entire study cohort (N = 1236)
Dose received	Extent to which participants received/used study components. Measurement focus is study participants	Caregiver message recall by treatment group	Caregiver exit interview (N = 219)
		Food compliance rates by treatment group	Entire study cohort (N = 1236)
Fidelity	Extent to which study was conducted as planned. Measurement focus is study staff	Percentage of home visits in which educational messages were delivered by community coordinators by treatment group	Entire study cohort (N = 1236)
Context	Larger environment that may have affected the study	Number of community coordinators identifying environmental, contextual or community factors by treatment group	Community coordinator exit interview (N=45)

We also developed a caregiver exit interview and a community coordinator exit interview to explore the mechanisms through which educational messages to optimize complementary feeding led to behavior change. Exit interviews were developed in English and translated into local languages.

Caregiver exit interview

A random sample of 20% of caregivers stratified by study site was selected from each treatment group to complete the caregiver exit interview. The exit interviews were administered to caregivers upon completion of the study in face-to-face interviews by a member of the assessment team, which was different from the intervention team (i.e. community coordinators). Constructs from the theory of planned behavior [11] were used to conceptualize reasons why caregivers may or may not have followed the educational messages and administered the study food to their children. The caregiver exit interview included seven items assessing behavioral beliefs, or beliefs that the behavior will result in an expected outcome (i.e. feeding my infant the study food will help him/her to grow). We included two items to assess normative beliefs, or caregivers’ beliefs about whether key people (i.e. family and other mothers) approved or disapproved of their feeding their infants the study food. Two items assessed the caregivers’ motivation to comply with family and other mothers’ opinions about how they take care of their infants. Five items were included to assess the caregivers’ perceived behavioral control, or the caregivers’ perception of their ability to perform a given behavior (i.e. prepare the study food, feed their infants the study food, feed solid foods, feed many types of food and boil water to prepare their infants’ food).

We also included 10 items to evaluate the dose received process evaluation indicator (i.e. the extent to which the caregivers received the messages delivered by the community coordinators during home visits). Caregivers were first asked to mention some of the things that the community coordinator talked to them about during home visits to assess message recall. To assess message recognition, caregivers were then asked whether the community coordinator talked to them about the following messages, if not already mentioned by the caregiver: (i) exclusive breastfeeding for the first 6 months; (ii) start complementary feeding at 6 months; (iii) how to prepare the study food; (iv) wash hands with soap before preparing infant’s food; (v) boil water before using it to prepare infant’s food; (vi) wash cooking utensils before preparing infant’s food; (vii) stop study food for 3 days if signs of allergic reaction and (viii–x) the three educational messages described above (thickened feeds, feeding frequency and encourage variety). In addition, we asked caregivers about their level of satisfaction with the study food, the home visits, and their participation in the study, about their overall perception of their infants’ health and whether their infants liked the study food.

Community coordinator exit interview

The community coordinator exit interview was administered to all coordinators during face-to-face interviews with study staff after they had completed their involvement with the study. The community coordinator exit interview was designed to explore reasons why caregivers may or may not have followed the educational messages and the contextual factors that may have influenced the study. To a large extent, the community coordinator exit interview was similar to the caregiver exit interview. Community coordinators were asked if it was easy or difficult to talk about each of the 10 educational messages that were included in the message recall/recognition section of the caregiver exit interview. Two items addressed the coordinators’ assessment of caregivers’ perceived behavioral control by asking whether it was difficult or easy for mothers to prepare the study food and feed their infants the study food. Three items addressed the coordinators’ assessment of caregivers’ behavioral beliefs about the three main educational messages described above (i.e. thickened feeds, feeding frequency and encourage variety).

The community coordinator exit interview also addressed contextual factors that may have influenced the study, such as natural disasters, civil unrest, strikes, media coverage about infant feeding or shipment of study materials. One of the goals of the global network is to build research capacity and enhance sustainability so coordinators were asked whether they thought their involvement in the study enhanced their professional development, improved their organizational, leadership and research skills and their ability to implement a behavior change intervention.

Data analysis

All analyses were performed using SAS 9.1 for Windows [12]. For analysis, the exit interview items addressing each construct were summed to create summed scores for behavioral beliefs, normative beliefs, motivation to comply and perceived behavioral control [11]. Chi-square tests were used to examine differences between treatment groups. Multivariate regression analysis was used to determine the relationship between caregiver scores and infant linear growth velocity.

Results

Characteristics of randomized groups at study start

We examined maternal and infant characteristics of the entire study cohort (N = 1236) to determine whether there were differences in the randomized groups at the start of the study that could explain differences in growth. There were no statistical differences between groups in socio-economic status (SES), maternal education, whether mother worked for pay and whether father worked for pay. However, the mean number of years of formal education was slightly higher for fathers in the cereal group as compared with fathers in the meat group (7.3 versus 6.4 years, respectively, P < 0.05). No differences were observed in the two groups for maternal characteristics such as height, weight, number of pregnancies, number of children, whether mothers breastfed their last child and whether mothers ever had malaria, tuberculosis, HIV, diabetes, anemia or parasites. Similarly, we did not observe differences in the two groups for infant characteristics such as prematurity, gender, birth weight, and whether children were exclusively breastfed at 6 months.

Process evaluation indicators

Reach

We examined early termination from the study by treatment group to assess reach, or the proportion of the target audience that participated in the study [10]. Of the entire study cohort (N = 1236), a total of 174 participants did not complete the study, 86 from the meat group and 88 from the cereal group. Reasons for terminating the study were similar between the meat and cereal groups: moved away from study site (28 versus 30), lost to follow-up (4 versus 3), study demands too high (21 for both groups), side-effect/adverse event (7 versus 8) and other (26 for both groups).

Dose delivered

We examined completion of home visits and study food distribution by treatment group among the entire study cohort (N = 1236) to assess dose delivered, or the proportion of the study components that was delivered to study participants [10]. We calculated the percentage of study activities completed by community coordinators of the total expected (Table II). Both the meat and cereal groups had high percentages of home visits completed as specified in the study protocol for the following study periods: daily (92% for both groups), three times per week (101% versus 100%) and weekly (117% versus 112%). Nearly, all 1236 participants in both groups (97% of meat group and 96% of cereal group) had weekly study food dispensed as specified in the study protocol.

Table II.

Study activities completed by community coordinators by treatment group

Variable	All sites combined
	Meat N (%)	Cereal N (%)	Total N (%)
Number enrolled	618	618	1236
Intervention home visit
Daily: number expecteda	8650	8745	17 395
Daily: number completed (%)	7955 (92.0)	8030 (91.8)	15 985 (91.9)
3× per week: number expecteda	15 330	15 477	30 807
3× per week: number completed (%)b	15 420 (100.6)	15 449 (99.8)	30 869 (100.2)
Weekly: number expecteda	22 195	22 243	44 438
Weekly: number completed (%)b	25 983 (117.1)	24 978 (112.3)	50 961 (114.7)
Weekly study food distribution
Number expected	29 143	29 143	58 169
Number completed (%)	28 065 (96.9)	28 065 (96.3)	56 185 (96.6)

^aAs per the study protocol, the frequency of required home visits decreased during the course of the study from daily to 3× per week to weekly.

^bPercentages >100% indicate more home visits were completed in the time period than were required by the study protocol.

Dose received

We examined caregiver message recall and food compliance rates by treatment group to assess the dose received, or the extent to which study components were received by study participants [10]. Caregiver message recall/recognition, assessed through the caregiver exit interview (N = 219), was high and exceeded 96% for both groups for all 10 educational messages except for stopping study food for 3 days if signs of allergic reaction occur (67% for meat group and 71% for cereal group), which was not significantly different between groups.

Food compliance rates were calculated by dividing the number of days the infant ate the study food by the number of study days. Compliance was monitored among the entire study cohort (N = 1236) at weekly visits by counting empty food packets and by parental report. The overall food compliance rate for both groups was 99%. A higher proportion of cereal group infant days without food as compared with meat group infant days without food was attributed to illness (38% versus 32%), lack of time (31% versus 14%) and did not like food (12% versus 3%), though differences between groups were not statistically significant.

Fidelity

We calculated the percentage of home visits among the entire study cohort in which educational messages were delivered by the community coordinators to assess fidelity, or the extent to which the study was conducted as planned [10]. As noted in Table II, compliance with the number of planned home visits and study food dispensing was very high. By design, the frequency of visits was reduced from daily to three times per week to weekly. Compliance with this schedule exceeded 90% for each frequency and did not differ by treatment group. Compliance with delivering weekly educational messages among the entire study cohort was also high. The message of feeding thickened food was given in 82% of the cereal group visits compared with 78% of the meat group visits. The message of feeding a wide variety of foods was given in 82% of both groups’ visits. The message of feeding solid foods at least three times per day was given in 86% of the cereal groups’ visits and 85% of the meat groups’ visits.

Context

We assessed environmental, contextual and community factors that may have influenced the study. The following were identified by coordinators (21 in the meat group and 24 in the cereal group) during exit interviews as factors that could have influenced the study: natural disasters (3 versus 8), civil/political unrest (2 versus 9), labor strikes (7 versus 9), increased media coverage about infant feeding (1 versus 2), another intervention/program in the community (3 versus 1) and shipment of study materials (3 versus 5). None of the differences between groups were statistically significant.

Caregiver exit interview

Exit interviews were completed by 219 caregivers (110 in the meat group and 109 in the cereal group). There were no differences between the meat group and the cereal group in caregiver message recall/recognition, satisfaction with the study food, satisfaction with home visits, caregivers’ perception of their child’s health, nor were there differences in behavioral beliefs, motivation to comply or perceived behavioral control.

There were modest differences between the meat and cereal groups in normative beliefs. When caregivers were asked whether their family thought they should feed their infants the study food, 93% of the meat group agreed while 99% of the cereal group agreed (P < 0.05). When caregivers were asked whether other mothers they know thought that they should feed their infants the study food, 89% of the meat group agreed while 97% of the cereal group agreed (P < 0.05).

Multivariate regression analysis was used to determine the relationship between caregiver scores and linear growth velocity (Table III). Treatment effect, behavioral belief score, normative belief score, motivation to comply score, perceived behavioral control score and message recall/recognition were all considered but only message recall/recognition was significantly associated with linear growth velocity when controlling for the other variables (P < 0.05). As message recall/recognition increased, linear growth velocity increased.

Table III.

Multivariate (GEE) analysis examining relationship between caregiver scores and linear growth velocity (cm/month)

Covariate	Main effect Estimate (95% CI)	P-value
Treatment group (meat)	−0.006 (0.035)	0.8663
Behavioral belief score	−0.109 (0.071)	0.1751
Normative belief score	−0.002 (0.023)	0.9415
Motivation to comply score	−0.010 (0.008)	0.2056
Perceived behavioral control score	0.025 (0.028)	0.4102
Message recall/recognition score	0.048 (0.019)	0.0304

Community coordinator exit interview

Exit interviews were completed by 45 community coordinators (21 in the meat group and 24 in the cereal group). There were no differences between the community coordinators assigned to the meat and cereal groups in behavioral beliefs, perceived behavioral control and ease of message delivery scores. All the coordinators in both groups indicated that they were satisfied with home visits, that they thought mothers were satisfied with the study food, that there were benefits of participating in the study for participants, and that overall, the study went well. Nearly, all (95% of the coordinators in the meat group and 100% of those in the cereal group) thought that the home visits were helpful to the mothers and that there were benefits of participating in the study for the coordinators.

Regarding coordinators’ perceptions of their professional development skills, all the coordinators indicated that their involvement in the study improved their organizational skills, leadership skills and professional development while nearly all the coordinators indicated that their involvement improved their research skills (98%) and ability to implement a behavior change intervention (96%).

Discussion

Including process evaluation alongside RCTs is helpful for understanding how, why and for whom interventions work or do not work [10]. Process evaluation was useful in examining pathways to success of an RCT to improve infant growth in Peru [13, 14], interpreting outcomes of an infant-feeding counseling protocol in Malawi [15] and assessing the implementation quality of programs to increase fruit and vegetable consumption among women [16] and children [17] in the United States.

Our detailed process evaluation revealed few differences between treatment groups which gives us confidence that the main trial’s lack of intervention effect [9] cannot be explained by differences between treatment groups or inconsistencies in protocol implementation. Our process evaluation revealed that as caregiver message recall increased, irrespective of treatment group, linear growth velocity increased.

Caregiver message recall has been found elsewhere to positively influence caregiver feeding behavior which is, in turn, thought to improve child growth outcomes [5, 6, 13]. A process evaluation of a nutrition education intervention in Peru [13] found support for a conceptual model of the pathway of improved infant growth [18] where health center implementation positively influenced caregiver exposure, which positively influenced caregiver message recall, and, in turn, caregiver feeding behavior. The positive association between caregiver message recall and improved growth is consistent with the current study’s main trial results in which maternal education was among the non-nutritional factors significantly associated with linear growth velocity [9]. Enhancement of literacy and language abilities resulting from increased education has been hypothesized to enable women to better understand health messages, better acquire knowledge overall and better navigate healthcare settings [19]. Furthermore, if schools are viewed as ‘transmitters of cultures’, women may internalize the teacher–student relationship in which they assume the role as teacher to their children in the household and the role of student when responding to instructions from healthcare providers and health messages [19]. Maternal education is commonly thought to be a proxy for SES though SES was not found to be a significant covariate of linear growth velocity in the models reported in the main trial [9]. Findings on maternal education are not presented here to fault mothers for their children’s poor outcomes but rather to highlight low maternal education as a potential risk factor so that interventions can be adapted accordingly. Our findings also underscore the importance of study messages. The positive association between caregiver message recall and linear growth velocity suggests that the negative impact of low maternal education could potentially be mitigated by adapting interventions in low-literacy settings. For example, others have demonstrated the importance of how messages are delivered in low-literacy populations, such as with visual cues and materials for low-literacy caregivers when planning educational interventions alongside complementary feeding interventions [5, 20].

Process evaluation factors identified a priori that potentially could have accounted for differences in the primary outcome (linear growth velocity) between groups included differences in caregiver characteristics (i.e. SES, health, beliefs, attitudes, message recall), infant characteristics (prematurity, gender, birth weight, whether child was exclusively breastfed at 6 months), intervention staff (i.e. beliefs, attitudes, message delivery) and differences between groups in key process evaluation indicators (i.e. reach, dose delivered, dose received, fidelity and context [10]). However, few differences between treatment groups were observed, which we interpreted favorably from a process evaluation standpoint.

Strengths and limitations

A strength of this process evaluation is our use of health behavior theory as a framework to conceptualize reasons why caregivers may or may not have followed the educational messages and administered the study food to their children. The use of the theory of planned behavior [11] coupled with process evaluation methodology [10] aided in identifying factors that may have influenced caregiver behavior, developing exit interview questions and assessing key process evaluation indicators [10], all of which were informative for considering factors that may have impacted the trial outcomes. A limitation of this process evaluation is that exit interview data were collected during face-to-face interviews with study staff, which could have led to social desirability bias.

Conclusions

Collecting data on theoretical constructs [11] and key process evaluation indicators [10] enabled us to explore potential competing explanations of the main trial’s findings. We observed few differences between treatment groups which gives us confidence in the validity of the primary outcome results. Although most caregivers and other mothers they knew thought they should feed their infants the study meat, the percentage was slightly lower than that for the cereal group, suggesting the need for community-wide education on the inclusion of a wide variety of complementary foods, including meats. The value of such education is substantiated by the significant positive association between message recall/recognition and linear growth velocity. Infants aged 6–18 months in this study almost universally accepted the meat study food with a greater proportion disliking the cereal study food.

The findings of the main trial, and the support of their validity provided by this process evaluation, add to an emerging body of literature that suggests limited impact on stunting of interventions initiated during the period of complementary feeding. Although other benefits may be realized from dietary improvement during this critical window, the very early onset and steady progression of growth faltering in these impoverished environments support the provision of earlier and multi-faceted interventions to avert its negative impacts.

Funding

This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development [HD040657 (University of Colorado Denver), HD043464 (University of Alabama at Birmingham), HD040607 (Drexel), HD043475 (University of North Carolina at Chapel Hill), HD040636 (RTI)]; Office of Dietary Supplements, and National Institute of Diabetes and Digestive and Kidney Diseases (9K24 DK083772).

Conflict of interest statement

Author Waldemar A. Carlo is on the Board of Directors, Mednax, Inc.