Process Evaluation of Community-Based Nutrition Service for Under 2 Years Children in Degadamot District Using Qualitative Approach, Western Ethiopia, 2023

Abstract

Malnutrition accounts for 45% of under-5 child morbidities in Ethiopia and is a more common issue in rural communities. In that regard, Ethiopia has implemented a CBN program at the community level to improve the nutrition status of children. Despite this, malnutrition still causes a significant rate of child morbidity and mortality. The implementation status of the CBN program is unknown. Thus, this evaluation aimed to evaluate the children under 2 years in Degadamot district health posts in northwest Ethiopia. A facility-based single-case study design with qualitative-method evaluation was employed from March 28 to April 28, 2023. The evaluation focused on the availability and compliance with multiple data sources. The qualitative data were transcribed, translated, and analyzed using thematic analysis with OPEN CODE version 4.02 software. Finally, the overall program implementation was determined based on predetermined judgmental criteria. According to the predetermined judgment parameter, the overall CBN program implementation level was 62.68%, measured by the availability of resources (71.36%) and compliance of service providers (54.0%). The overall community-based nutrition program implementation in the Degadamot district health post with 2 evaluation dimensions was judged as fair based on the presetting criteria. It is better to improve the service through training HEW and HDA, distributing nutritional supplies, providing regular support, and following up with HEWs for GMP performance in the Degadamot district.

Introduction

The first 1000 days of life, starting from the onset of pregnancy until the child reaches 24 months of age, is a critical window for healthy growth and development. During this period addressing the nutritional needs of children has a significant effect on their healthy development, behavior, and intellectual advancement. ¹ Malnutrition is characterized by undernutrition or overnutrition resulting from unbalanced intake of vital nutrients, or poor nutrient utilization. Undernutrition, manifests with Stunting, wasting, being underweight, and micronutrient deficiencies. ² Malnutrition in children is one of the major global health problems, leading to childhood morbidity, and mortality, hindered cognitive growth, reduced adult productivity, and increased susceptibility to illnesses later in life.^2,3 Community-based nutrition service is one component of a national nutrition program that aims to build up the capacity and ownership of communities and families to make informed decisions on child nutrition practices. ⁴

Community-based nutrition service is focused on children under 2 and aims to improve their nutritional status by enhancing the performance of Health Extension Workers in Growth Monitoring and Promotion activities by using the “triple A” approach (Assessment, Analysis, and Action). ⁵ Assessing problems in a child’s growth through, weighing a child regularly, and comparing the child’s growth with the standard and with their previous weights, followed by Analyzing its causes by looking into issues of feeding practice and quality. Finally based on the analysis Action includes providing nutrition education and communications for behavioral change on how to improve feeding, follow up on immunization, and deworming is taken.^3,5 Therefore, engaging multi-sectoral support is necessary to improve children’s nutrition practices and caring behaviors, as well as community-based nutrition services to overcome malnutrition that occurs through inappropriate feeding. ⁶

Global evidence shows that child malnutrition is a result of poor nutrition practices, especially in the first year of life, which increases the risk of death from acute respiratory infection, diarrhea, malaria, measles, and other infectious diseases. ⁷ In 2020, globally, 149.2 million children under the age of 5 were stunted, 45.4 million, were wasted, and 38.9 million were overweight. Some parts of the world experience undernutrition more than others. Prevalence among children with stunting is declining in all regions except Africa. South Asia and sub-Saharan Africa as a whole had more than half and a quarter of the world’s wasted children, respectively. ⁸ To address this issue, the World Health Organization (WHO) and UNICEF designed and implemented infant and young child feeding (IYCF) strategies. ⁹ However, poor feeding practices such as suboptimal breastfeeding, low meal frequency and dietary diversity, prenatal iron supplementation, socio-demographic and environmental factors, and health status continue to be strong determinants of undernutrition.^{10 -12} Malnutrition accounts for 45% of under-5 child morbidity in Ethiopia and is a more common issue in rural communities. ¹³

The government of Ethiopia has implemented different nutritional interventions such as community-based nutrition to address the high burden of undernutrition and has a strong commitment to accelerate the reduction of stunting and end child malnutrition by 2030. However, despite the initiative, more than 5.8 million children under 5 are affected by stunting, and 1.2 million are affected by wasting.^14,15 In the Amhara region, the prevalence of stunting, wasting, and being underweight among children under the age of 5 was reported as, 42%, 8%, and 28%, respectively. ¹⁶ The lack of progress in the reduction of undernutrition is explained by low availability of nutrition services and poor nutrition utilization by mothers.^17,18 Previous evaluation studies on community-based nutrition intervention indicate that there is a gap in reaching mothers with service. ¹⁹

Description of a Community-Based Nutrition Program

Program Stakeholders and Community-Based Nutrition Program Logic Model

The major stakeholders that were identified during the evaluation assessment are those involved in direct program operation and implementation, those served or affected by the program, primary users of the program, and those who have an interest in the program. These are Amhara, the regional health bureau, the West Gojjam zone health bureau, the Degadamot district health coordinator, the health center health worker in Degadamot district, UNICEF, HEW, the service user (under 2-year children mother), HDA, and communication is through phone and in-person communication displayed in Table 1 and Figure 1.

Table 1.

The Stakeholders’ Analysis Matrix of Community-Based Nutrition Service for Under 2-Year Children in Degadamot District, 2023.

Stakeholders	Role in the program	Interest/perspective on evaluation	Role in the evaluation	Communication strategies
Regional health bureau	Funding, supply, & training provider	Find user	Evaluation fund provider	Formal letter
District health office coordinator	Planning coordinating and facilitating Resource allocation	Evaluation results used for the decision to improve the program	coordinate evaluation Information Source Evaluation questions Identify & assign a weight for indicators	Formal letter Phone Interview
HC health worker	Program supporters Strength linkage of PHCU	Evaluation results used for technical support	Information Source Identify indicators Formulate evaluation questions	Phone Interview
UNICEF	Support Fund & supplies, provide training	Evaluation results users to identify the support needed areas	Information sources Evaluation planning Evaluation question	Formal letter Email
HEWs	Program implementor	To identify the gap in implementation	Information Source use finding	Phone Interview
U2, child mother	Primary program beneficiaries	Program improvement	Information source	Interview
HDA	Community mobilization	To know the level of implementation	Information source	Interview

Figure 1.

Logic model for community-based nutrition services for under 2 years children in Degadamot district 2023, adapted from CBN guidelines. ²⁰

Stage of Program Development

Global Situation

The United Nations Children’s Fund and WHO organized a consultation in Geneva in March 2000 to assess infant and young child nutrition practices by reviewing key interventions. And also formulate a strategy that promotes appropriate feeding for infants and young children. Appropriate evidence-based feeding practices are essential for attaining and maintaining proper nutrition and health. ²¹

Ethiopia Situation

Community-Based Nutrition, as a preventative measure, was implemented to build upon the Health Extension Program to reduce undernutrition through preventative actions in 238 woredas in the 4 main regions of Ethiopia: Amhara, Oromia, SNNP, and Tigray in 2008. The initiatives include community dialogs, referral networks, micronutrient deficiency management, and growth monitoring and promotion for children under 2 years old. ²⁰ Then, implementation expanded to 386 woredas in 2014, covering approximately half of all woredas in the country. An estimated 59 million people are considering addressing the highest stunting and underweight prevalence rates in Ethiopia with the support of UNICEF and development partners. ²²

The program promotes and provides monthly growth monitoring and promotion for under 2-year-old children, treatment, and counseling to mothers in the community to improve the nutritional status of children by understanding obstacles and possible support for enhanced nutrition levels, primarily through the Health Extension Program, by utilizing the large network of HEWs and community volunteers. ¹⁹

Degadamot District Situation

In the study area, Degadamot district, community-based nutrition for under-2 children program implementation has started in 32 rural kebele health posts since 2008 G.C. Currently, community-based nutritional programs in the implementation stage provide an opportunity to improve other Maternal, Newborn, and Child Health (MNCH) programs and strengthen the health system through health extension programs with the community.

Evaluation Methods

Study Area

The evaluation study was conducted in the Dega Damot district, which is found in the West Gojjam Zone in the Amhara region, which is 410 km to the northwest of Addis Ababa and 275 km from Bahir Dar City. It is mountainous with high land and low land climate and produces different crops. The most commonly used foods in the highland are potato, barely, and peens while lowland residents, commonly consumed maize, teff, and barley foods. Erosion is very high in this area.

According to the 2022 district health office report, the district has an estimated total population of 189 806. Lactating mothers with children 0 to 23 months old are 6523, and under-5 children are 13 685. The district has 34 kebeles (2 kebeles urban and 32 kebeles rural) 34 health posts with 105 health extension health workers, 8 health centers, 1 primary hospital, 2 private clinics, and 1 private drug store. ²³

Evaluation Period

The evaluability assessment was conducted from February 1-5, 2023, with the Woreda nutrition focal officer and HEW. The data collection for the study was conducted from March 28 to April 28 in 10 health posts in Degadamot district, west of Gojjam Zone, Ethiopia, in 2023.

Evaluation Design

This evaluation tries to assess a comprehensive picture of the real-environment context and understand different aspects of program implementation in detail for future improvement. With these characteristics design a case study design was chosen.

A facility-based single case study with qualitative method was used to evaluate community-based nutrition services for under 2 years children.

Evaluation Approach

Since community-based nutrition service is an ongoing service, a formative evaluation approach was conducted to understand the nature of implementation, determine the extent of service, and provide relevant information for program stakeholders and decision-makers to improve the implementation of community-based nutrition services being provided in the Degadamot district health post.

Focus of Evaluation and Dimensions

Focus of evaluation

The focus of the evaluation was on the process components of the community-based nutrition program input, activities, and output that are used to explain the implementation status of CBN for children under 2 years.

Evaluation dimensions and framework

The evaluation used availability ²⁴ dimensions from access and work, and compliance from implementation fidelity. ²⁵

The above-mentioned dimension was chosen over the others with the best knowledge of the principal evaluator regarding the relevance of answering evaluation questions raised by stakeholders with limited.

Assessing the availability of required resources is the base for the starting point to facilitate the capacity of service to meet the objective of community-based nutrition service for children under 2 in the district.

Assessing Compliance of HEW to national guidelines is an important parameter to improve community-based nutrition services for under 2 years children. Poor compliance of HEW with the program context results in low performance to reach the program goal. The consciousness of adherence is important to identify and enhance the implementation of services. ²⁶ Even if community-based nutrition services implemented so far, insight into HEW compliance is limited in the Degadamot district.

Indicators and Variables

Indicators

During the evaluability assessment, indicators were identified through the active participation of stakeholders based on their relevance to the program together with the principal evaluator. The indicators are adapted from the national CBN implementation guideline and related literature.^{14,27 -29}

Availability indicator: Proportion of HPs with trained HEWs on community-based nutrition service, proportion of HPs with trained HDAs on community-based nutrition service, proportion of HPs with a functional weight scale, proportion of HPs having standard GMP registration books, proportion of HPs with MUAC tapes, proportion of HPs with child health growth charts (weight-for-age tables), Proportion of HPs with job aids for BF support and food demonstration leaflets, proportion of HPs with a quick nutritional status classification reference book, proportion of HPs with a monthly reporting format, proportion of HPs with vitamin A capsules.

Compliance indicator: proportion of sessions that HEWs show respect for children’s mothers (greeted and offered seats), proportion of 6 to 23-month children for whom WFA was performed by growth chart, proportion of children who identified their nutritional status with the standard growth chart, proportion of mothers who were oriented about the next GMP session, proportion of identified SAM cases with MUAC and referred to the next level, proportion of children aged 6 to 23 months who received vitamin A supplementation according to guidelines, proportion of breast-feeding mothers who were counseled by HEW, proportion of children checked for edema status by HEW.

Operational Definition

Availability: This dimension assessed the physical existence of health post resources to provide community-based nutrition services for under 2 children. According to this evaluation, availability was measured with 10 indicators weighting 30% and considered to be ≤59% poor, 60%-74.9%, fair; 75%-84.9%, good; and ≥85%, V. good. ³⁰

$$Availability=\frac{\begin{array}{l}sumtotalofobservedresult\\ underavalibilityindicator\end{array}}{weightavalibilitydimension}\times 100$$

Compliance: according to this study, compliance assessed the adherence of HEW with the national guidelines in providing community-based nutrition services for under-2 children being implemented in the Degadamot district. And it was measured with 8 indicators weighting 35% and considered ≤59%, poor; 60%-74.9%, fair; 75%-84.9%, good; and ≥85%, V. good. ³⁰

$$\mathrm{Compliance}=\frac{\begin{array}{l}sumtotalofobservedresult\\ undercomplianceindicator\end{array}}{weightcompliancedimension}\times 100$$

Overall implementation status CBN service for children under 2 years

According to this study, the overall implementation status was measured with a total of 30 indicators with their weight under availability, and compliance judged based on the judgment parameter as ≤59% poor, 60%-74.9% fair, 75%-84.9% good and ≥85%-100% very good implementation. ³⁰

$$\begin{array}{l}\mathrm{Overall}\mathrm{implementation}\mathrm{status}=\\ \frac{\begin{array}{l}\mathbf{S}um\mathbf{t}otal\mathbf{o}f\mathbf{a}chivement\\ \mathbf{s}core\mathbf{u}nder\mathbf{e}ach\mathbf{d}imension\end{array}}{\mathbf{T}otal\#\mathbf{o}f\mathbf{d}imension\mathbf{w}eight}\times 100\end{array}$$

Sample Size Determination

Retrospective document reviews and observations were used to measure the compliance of HEW service delivery with the guidelines. Documents from 20% of 1 month’s GMP registries(10%-20% of the eligible charts per variable applicable minimum sample size in patient safety and quality improvement ³¹ were reviewed file before data collection time in each health post. It includes a review of records or reports for children under 2 years of age.

A total of 50 non-participatory observation sessions (five observations per HEW in each health post) were conducted to gather information about HEW and mother-child interaction. A total of 3 to 5 observation sessions is enough for each service provision or counseling unit. ³²

To assess the availability of required supplies, materials, and human resources for the program, ten randomly selected health posts (30% of the total health posts in the district according to the WHO recommendation for the assessment of the operationality of the district health system ³³ were included in the evaluation.

Key informant interviews: A purposeful sampling technique was used to select key informants based on responsibility and experience. Eight key informants 1 head of health office, 1 woreda CBN focal person, 2 health center nutrition focal persons, and 4 health extension workers were selected purposefully and interviewed. The sample size of the interview was determined with the information saturation point.

Sampling Procedure/Technique

Document review

Since the program document has no individual folder for children’s nutritional status. But each file is exclusively registered in the GMP registration book within their MRN, so a survey of 20% of the 1-month under-2-years children’s registration book was conducted.

Observation

Fifty non-participatory observations (5 observations per HEW in each health post) were conducted using opportunity sampling at the time of arrival at HP and then consecutively.

Key informant interviews

A purposeful sampling technique was conducted to select key informants based on responsibility and experience to address why and how questions and to explore the gaps for program implementation. Those who were information-reach included to get detailed and in-depth information.

Data Collection

Development of data collection tools

Structured observation check lists, document reviews, and semi-structured questionnaires for key informants were used. The questionnaires were adapted by referring to different literature, a national community-based nutrition guideline, and a checklist.^19,34,35

The interviewer-administered questionnaire was adapted by reviewing relevant literature related to mothers’ acceptability of community-based nutrition in under-2-year children.^15,34 The questionnaire was developed in English and then translated into Amharic, the local language, to ensure the consistency of the questions.

The key informant’s interview guide, to find out the availability of resources for community-based nutrition programs and delivery barriers to program implementation and other recommended solutions developed by reviewing the literature. ¹⁴

An inventory checklist containing community-based under-2-year-old children’s nutrition services, human resources, supplies, and logistics (guidelines, recording, and reporting tools). These tools were used to assess the availability of program resources and were adapted from national guidelines for nutrition programs. ¹⁴

An observation checklist was prepared to assess the compliance of HEW while delivering community-based nutrition services (client-provider interaction). It was adapted by referring to the national supervision checklist and guidelines.^27,36,37

Document review checklists were prepared by reviewing different literature and community-based nutrition program guidelines. ²⁷

Data collectors

Seven ⁷ recent graduate BSc professional health workers, supervised by 1 supervisor and the principal investigator. Non-participatory observations were collected by the principal evaluator, together with 1 Master’s degree holder in public health from another facility and key informant interviews conducted with the principal evaluator.

The data collectors were trained on the content of the data to collect, ethical issues to be addressed, and how to use the data collection guide and tools of the principal evaluator for 1 day. Supervisors are trained on the content to be covered during data collectors training, on how to manage the data collection process, and on how to monitor the quality of data by the principal evaluator.

Data collection fieldwork

Non-participatory observation data was collected from ten selected health posts by the principal evaluator, together with 1 Master’s degree holder in public. Before starting data, collection consent was received from both the health extension worker and the client. Finally key informant interview was conducted with 8 program personnel by the principal evaluator. Before starting an interview, consent was received from the health extension worker.

Data Management and Analysis

The qualitative data, field notes were written as fair notes, and key informant interviews were properly recorded and transcribed to the Open Code Version 4.02 software for analysis and thematic analysis. Then the inconsistent, data was refined appropriately to get the maximum quality of data before, during, and after data entry. The results of the analysis were compared with a predetermined evaluation judgment matrix to determine the overall level of implementation. All records of the key informants were properly recorded and transcribed verbatim into Amharic, then translated into English. To use thematic analysis techniques, the principal investigator has to reread descriptive information to be near facts to obtain a code for thematic analysis in theme- and sub-theme-based predetermined evaluation questions. The translated data was coded with Open Code Version 4.02 software for analysis, then the analysis result was 37 codes categorized into 7 subthemes and 3 main themes using evaluation dimensions using thematic analysis techniques, and finally, a meaningful report was written.

Matrix of Analysis & Judgment Parameter

The judgment parameters for availability with 10, compliance with 8 indicators were developed after reviewing different sources. The weighted values for availability and compliance dimensions were given based on stakeholder and evaluator agreement on each indicator.

The cutoff point is set by the combination of stakeholders’ agreement and an evaluation study on the implementation of the OTP service in northern Ethiopia. ≤59%, poor; 60%-74.9%, fair; 75%-84.9%, good; and ≥85%, very good implementation. ³⁰ Matrix of analysis & judgment parameter of this study as shown in Table 2.

Table 2.

Overall Judgment Matrix and Analysis of CBN Services for Children Under 2 Years in Degadamot District, West Gojjam, Ethiopia, 2023.

Dimension	Indicator	Weight	Expected value %	Achieved in %	Judgment parameter
Availability	10	56%	100%		≥85-100 = Very Good 75%-84.9% = Good 60-74.9% = Fair <60 = Poor
Compliance	8	44%	100%
Overall judgment	18	100%	100%

Note. Very good = successful implementation; Good = partially implemented needs improvement; Fair = fairly implemented need urgent improvement; Poor = not successfully implemented need urgent and major improvement.

Result

A total of 10 HPs for resources inventors, A total of 30 non-participatory observations (3 in each health post), 20% of 1 month’s retrospective document review, and 8 key informant interviews (4 in health office staff and 4 in health posts) were performed.

Availability of Program Resources

This part of the evaluation result depicts the findings of program resources that were available during the evaluation to deliver community-based nutrition services.

Availability of Human Resources and Training Status

Human resources include staff in the facilities, staff trained and knowledgeable about nutrition services, and support for staff to implement community-based nutrition services.

In the Degadamot district health facility, there were a total of 105 HEWs and 137 healthcare workers. Regarding the training and preparation of health care providers, a total of 33.3% of health center health workers, 25% of office staff, and 68.6% of HEWs were trained on community-based nutrition this year (Table 3).

Table 3.

Human Resources Availability and Training Status for CBN Services for Children Under 2 Years in Degadamot District Health Facilities April 2023.

Human resources
Professional category	Total available	Trained
MPH	8	25%
Public Health officer	29	0%
Nurse & Pharmacy	72	33.3%
Laboratory	20	0%
In ten health post
HEW	32	68.6%
HAD	212	23.6%

A total of 32 HEWs and 212 health development army members were available in ten health posts. HEW has had community-based nutrition training in different opportunities before this year, but only 23.6% of health development army members in the health post catchment have gotten orientation on community-based nutrition services which was judged as poor.

Key informant interviews results supported this finding, community-based nutrition training had been given, but not as frequently as other program training.

I have worked in this district health post for 12 years. I got community-based nutrition training in the last year but not in this year, but from my staff, two HEWs got training (35 years old, HEW).

I have 6 years of work experience in this health post, and I have received community-based nutrition 1-day orientation this year due to the campaign work. Nutrition training is not given as much as other training in our district (26 years old, HEW).

. . . HEW is not supported by non-governmental organizations, as compared to the last 2 years. 2 years ago, the health development army received refreshment training in Kebele with the support of UNICEF; currently, the training has not been given to the health development army and has not been actively supported for us (26 years old, HEW).

Availability of CBN Service Assessment Tool and Equipment

In this evaluation equipment availability and functioning against standards were assessed using the checklist. Functional weighting scale, and mid-upper arm circumference (MUAC) tape were 100% available. MUAC classification sheets, registration books, and monthly report forms were fully available. Regarding the community-based nutrition guidelines, growth charts, and client nutrition education and counseling materials (IEC/BCC leaflets) posted in health posts were 30%, 20%, and 70% available respectively but child-length boards availability was 0% (Table 4).

Table 4.

Availability of CBN Services Assessment Tool and Documents for Children Under 2 Years in Degadamot District, Gojjam, Ethiopia 2023 (n = 10).

Assessment tool and document	Available in the HPs during data collection
	Functional during evaluation	(n = 10)	Availability %
Registration book& report format	✓	10	10 (100%)
Quick nutrition references book	✓	3	3 (30%)
MUAC Classification	✓	10	10 (100%)
IEC/BCC material	✓	7	7 (70%)
Height or length measurements		0	0%
MUAC tape	✓	10	10 (100%)
Weight Scale	✓	10	10 (100%0
WHO, child standard Growth Chart (WFA)		2	2 (20%)
Water for drinking	✓	10	10 (100%)
Average			72.2%

This idea was supported by a key informant interview, the reason for being unavailable was due to a lack of commitment by health extension workers, especially to perform weight-for-age (WFA) for children,

. . .Even though HEW has the training to perform WFA, currently HEW measures the weight of children monthly and compares from last month’s weight, but WFA is not calculated. They mostly perform nutritional classification of children by measuring mid-upper arm circumstances (28-year-old Male Health Center nutrition focal).

Availability of CBN Service Drug and Supply

In observed health posts, the resources inventory revealed that vitamin A and RUTF were 100% and 30% available respectively (Figure 2).

Figure 2.

Availability of CBN services drug and supply for children under 2 years in Degadamot district, Gojjam, Ethiopia, 2023.

The key informant’s interview result supported this finding, in most health posts, RUTF was not available due to,

There is a shortage of plump nuts and other drugs because HEW did not send the request report form from the health post on time and They are asking us after finishing what they have on their hands (34-year-old Male nutrition team leader).

In summary, according to the predetermined judgment parameter, the process of the community-based nutrition program in Degadamot health post concerning resource availability was measured at only 71.36%, which was fair (Table 5).

Table 5.

Summary of the Judgment Analysis Matrix of the Availability Dimension for the Evaluation of CBN Service for Children U2 Years in D/D District, W/Gojjam, Ethiopia, 2023.

Indicator	E* #	O *#	W*	S*	A *%	JP*
Proportion of health posts with trained HEWs on community-based nutrition service	105	72	3.3	2.26	68.6	Fair
Proportion of HDA trained on community-based nutrition service in the health post catchment area.	212	50	2.8	0.66	23.6	Poor
Proportion of HPs with a functional weight scale.	10	10	3.1	3.1	100	V.Good
Proportion of HPs having standard GMP registration book	10	10	3.0	3	100	V.Good
Proportion of health posts with MUAC tapes	10	10	3.0	3	100	V.Good
Proportion of HPs with child health growth charts (Weight-for-age tables).	10	2	3.0	0.6	20	Poor
Proportion of HPs with job aid for BF support and food demonstration leaflets.	10	7	2.7	1.89	70	Fair
Proportion of HPs with quick nutritional status classification reference book	10	3	3.0	0.9	30	Poor
Proportion of HPs with monthly reporting format.	10	10	2.9	2.9	100	V.Good
Proportion of health posts with vitamin A capsules	10	10	3.1	3.1	100	V.Good
Overall availability of CBN by mother			30	21.41	71	Fair

E* = Expected, O* = Observed, W* = Weight, S* = Score = (Observed × weight)/Expected), A* = Achievement in percentage = (S/W) × 100), JP* = Judgment Parameter.

Most key informant findings stated that to improve the activities of community-based nutrition services in the community, districts need to have continuous financial and technical support for the health post.

. . .If the districts need continuous budget support for the health extension services according to the report and community outreach activities and coordinate regular linkages between the health center and the health post, it will be possible to improve the nutrition service (38-year-old HEW).

Compliance Health Extension Workers to the National Guideline

In this study, a total of 373 under-2-year-old GMP registration book files were reviewed. In the reviewed GMP registration book, 100% of the children’s basic identification information (age, sex, weight, and MUAC) was documented correctly (Figure 3).

Figure 3.

Compliance of HEW during the evaluation of CBN service for under two children in Degadamot health post, 2023.

From the document review, it was found that 98.12% of children’s nutritional status was classified with MUAC in the registration file in line with guidelines. From the MUAC results, 0.82% were less than 11.5 cm, which referred to the next health facility, and 3.0% were between 11.5 and 12.5 cm; the rest is normal status. Regarding vitamin A supplement status, about 94.37% of children were registered in the registration book.

Compliance HEW to Nutrition Education and Client Interaction

In this evaluation, observation findings revealed that more than half (53.3%) of mothers have gotten great service from health extension workers. Regarding providing nutrition education using nutrition education materials for mothers (brochures, posters), the direct observation finding revealed that only 40% of clients received counseling information about appropriate infant and young child feeding practices on breastfeeding and a balanced diet, but it was not checked whether the mothers understood that information (Figure 3). Seventy percent of the mothers received appointment counseling on when to return to the next service. The key informant’s result supported this finding stating that maternal education was performed at Kebele Center.

We provide nutritional education by getting mothers who have children into the kebele center, teaching them how to breastfeed their children, and showing food cooking demonstrations on how to prepare food from different types of grains that are locally available (35-year-old, HEW)

. . .Community-based nutrition services started seven days after the babies’ delivery. We go home to home and provide post-natal care. During this time, we advise mothers on how to breastfeed and when to start complementary foods, as well as how to feed them (30-year-old, HEW).

 Compliance HEWs to Children Nutritional Status Assessment

In this evaluation, direct observation revealed that nearly three-fourths (73.3%) of the children were correctly assessed for edema status by HEW using the appropriate technique.

Regarding nutritional assessment through measuring weight, 100% of children were correctly weighed using the appropriate equipment by HEW and communicated the results to the mother.

In this evaluation finding, growth monitoring, and promotion performance for under 2 years of children, using the WHO child growth standard reference chart was 0% (Figure 3). This finding supported by qualitative results, revealed that,

. . .We record all activities and performances we have done, then report them to the health center. We measure the weight and MUAC of children, but WFA doesn’t calculate because this is calculated in the health center (26-year-old, HEW).

We have done GMP monthly, MUAC every 6 months, and we observe children’s status. We also provide home visits in outreach services activities, and mothers are happy with the services we provide (35-year-old, HEW).

Based on observation and community-based nutrition document review, the overall compliance of health extension workers was found to be 54.0%, which was poor according to the predetermined judgment parameter (Table 6). To improve the compliance of HEW, one of the key informants stated that

Table 6.

Summary of Judgment Analysis Matrix of Compliance Dimension for the Evaluation of CBN Service for Children Under 2 Years in D/D, Gojjam, Ethiopia, 2023.

Indicator	E*	O*	W*	S*	A*	JP*
Proportion of sessions HEWs showed respect for the children’s mother (greeting and offering seats)	30	16	4.25	2.26	53.3%	Poor
Proportion of 6 to 23-month children whom WFA performed was performed with a growth chart.	373	0	4.4	0	0	Poor
Proportion of Children who were identified about their nutritional status according to the WFA growth chart.	373	0	4.5	0	0	Poor
Proportion of mothers who oriented about the next GMP session.	30	21	4.25	2.98	70%	Fair
Proportion of Children identified nutritional status accordingly with MUAC	373	373	4.2	4.2	100%	V.Good
The proportion of children aged 6 to 23 months who receive vitamin A according to guideline	373	373	4.5	4.5	100%	V.Good
Proportion of breast-feeding mothers who are nutrition counseled by HEW	30	12	4.7	1.88	40%	Poor
Proportion of children checked for edema status	30	22	4.2	3.08	73.3%	Fair
Overall compliance of CBN by HEW			35	18.9	54%	Poor

E* = Expected, O* = Observed, W* = Weight, S* = Score = (Observed × weight)/Expected), A* = Achievement in percentage = (S/W) * 100), JP* = Judgment Parameter.

We discuss with the health office nutrition team leader how to enhance compliance with HEW by organizing a health worker team at the health center level to support a checklist for basic HEW activity and to identify as well as fill gaps. In addition, every 3 months, an integrated community-based service supervision and review meeting is conducted for health extension performance in the woreda to plan versus achievement evaluation and set a direction or solution for solving the problems that are seen and raised during evaluation (38-year-old, District health office staff).

Meta-evaluation

The meta-evaluation was performed by the principal evaluator based on the checklist adapted from program evaluation standards(utility with 7 standards, propriety with 8 standards, feasibility with 3 standards, and accuracy with 12 standards,) set by Daniel L. Stufflebeam and Social Impact. ³⁸ For each of the 30 standards, the checklist includes 10 checkpoints drawn from the substance of the standard. Then judgments about the adequacy of the subject evaluation in meeting the standard can be: 0-2 Poor, 3-4 Fair, 5-6 Good, 7-8 Very Good, and 9-10 Excellent. The judgment parameter was decided to be excellent (93%-100%), very good (68%-93%), good (50%-68%), fair (25%-50%), and poor (25%).

Utility

Evaluation findings were determined by the information needs of stakeholders throughout the evaluation process. The stakeholders participated in the evaluation mainly as sources of data and facilitators during the community-based nutrition service evaluation data collection period. Therefore, the information needs and values of stakeholders were addressed to ensure the utilization of information. The utility standard is measured by 7 sub-standards and 23 checkpoints. 18 checkpoints fulfilled these specific criteria, and the final result was 78% according to the judgment parameter. The utility of these evaluations was very good.

Propriety

The data collector was trained on the content, how to deal with the evaluation, ensuring legal and ethical issues, and maintaining confidentiality. Approval of ethical clearance was obtained from the University of Gondar College of Medicine and Health Sciences and a support letter from the Department of Health Systems and Policy. Throughout the evaluation process, the consent of the participants was respected, and consent was taken both orally and in written form. The findings have been honest and not distorted in any way. The propriety standard is measured by 8 sub-standards and 20 checkpoints. 17 checkpoints fulfilled these specific criteria, and the final result was 85% according to the judgment parameter. The utility of these evaluations was very good.

Feasibility

The cost was considered due to the presence of limited resources and the resources that were used for the benefit of program improvement. The feasibility standard is measured by 8 criteria. Six checkpoints fulfilled these specific criteria, and the final result was 75% according to the judgment parameter. The utility of these evaluations was very good.

Accuracy

The evaluator reviewed all data collection, analysis, and presentation techniques that were carried out based on the scientific procedure. Data was collected from multiple sources, and triangulation was employed to strengthen the decision, convey technically adequate information about the feature, and determine the worth and benefit of community-based nutrition services. The accuracy standard is measured by 7 sub-standards and 26 checkpoints. 21 checkpoints fulfilled these specific criteria, and the final result was 80% according to the judgment parameter. The utility of these evaluations was very good (Table 7).

Table 7.

Summary of Meta-Evaluation Standards Result Using Specific Criteria Checklist for Process Evaluation of CBN Program in Degadamot District 2023.

Standards	Sub-standard	Checkpoint	# checkpoint met	%
Utility	Stakeholder identification	4	4
	Evaluator credibility	3	3
	Information scope and selection	3	2
	Values identification	3	2
	Report clarity	4	2
	Report timeliness and dissemination	3	2
	Impact evaluation	3	3
	Sub-total	23	18	78
Accuracy	Reliable information	3	3
	Systematic information	2	1
	Valid information	3	2
	Analysis of quantitative information	2	2
	Analysis of qualitative information	2	2
	Justified conclusion	2	2
	Impartial reporting	2	1
	Context analysis	2	1
	Program documentation	2	1
	Defensible information sources	2	2
	Described purposes and procedures	2	2
	Meta-evaluation	2	2
	Sub-total	26	21	80
Feasibility	Practical procedures	3	3
	Political viability	3	2
	Cost-effectiveness	2	1
	Sub-total	8	6	75
Propriety	Services orientation	4	3
	Formal agreement	3	2
	Rights of human subject	3	3
	Human interaction	2	2
	complete and fair assessment	2	2
	Disclosure of findings	3	3
	Conflict of interest	2	1
	Fiscal responsibility	1	1
	Sub-total	20	17	85
Total judgment parameter		77	64	83

Discussion

The implementation status of community-based nutrition programs was evaluated based on availability and compliance dimensions. The overall implementation status was judged as poor (<60%), fair (60%-74.9%), good (75%-84.9%), and very good (>85%). The cutoff points were set during the evaluability assessment, in combination with stakeholders’ agreement and Scientific findings. This evaluation thesis assessed the community-based nutrition program service implementation which was found fair (62.68%) according to a presetting parameter. The availability of program resources was fair (71.36%). However, HEWs’ compliance with the treatment guideline was judged to be poor (54%) as a presetting parameter. The overall judgment of the availability of CBN service-required resources was judged fair (71.36%). This evaluation finding is lower than the resource in the OTP implementation study in the Dehana District of Northern Ethiopia found that 87.5%. ³⁰ This evaluation finding was lower compared to the study conducted in the Hadiya Zone, which indicated that 85.7% of the required resources were available for community-based nutrition service provision. ³⁹ This finding was supported by qualitative findings in which currently HEWs are not supported by NGOs, as compared to the last 2 years, training and other material refills have not been actively supported. The other possible reason for this difference was the stockout of drugs and inadequate nutritional assessment guidelines and equipment. ³⁰

In this evaluation, the availability of nutrition guidelines, growth charts, and length boards was 30%, 20%, and 0% available for CBN service. The availability of community-based nutrition guidelines and nutrition assessment tools is predetermined in the national guidelines as one of the minimum requirements for the initiation and sustained delivery of community-based nutrition services at the health post level. ¹⁴ This finding lower than the study done in Uganda in 2018 which was found that all health facilities had basic nutrition assessment equipment (weighing, length/height measuring scale, MUAC tape), and essential job aid. ⁴⁰ This might be improper distribution of logistics, which might delay children’s nutrition screening, which increases complications and requires additional costs for further health care systems. ⁴¹

Regarding human power and training status, about 68.6% were trained in community-based nutrition services in the 2015 EFY budget year. This finding was less than the study on the HEW training status and perceived competency on the health extension package activities at the national level, which indicated that 75% of the health extension package activities were confidentially performed by the HEWs. ⁴² This variation was supported by qualitative findings in which community-based nutrition training had been given, but not as frequently as other program training. A possible explanation might be the allocation of an inadequate budget for training purposes at the woreda level.

In this evaluation finding, required resources such as a weight measurement scale, MUAC tape, MUAC classification sheet, and vitamin A were 100% available. This finding was greater than the community-based cross-sectional study conducted in the Benishangul Gumuz region, which found that 76% of health facilities had the necessary MUAC tapes, reporting formats, and registration books on hand for program implementation. ⁴³ The possible reason for this variation might be the availability and good utilization of these program resources by health extension workers. On the other hand, the availability of job aids (IEC/BCC) leaflets posted in the working area, was 70% of health posts, in Degadamot district, which was below the expected minimum standard according to national guidelines. The findings were higher than the cross-sectional study conducted in 2016 in the Enessie district, North Eastern Ethiopia, which indicated that on average 44.1% of health facilities were available. ⁴⁴ The possible reason for this difference might be the study design and time for the advancement of infrastructure.

The availability of RUTF in the working area, was in 30% of health posts, in the district which was judged as poor. This finding was lower than the study in rural Southern Ethiopia on community nutrition, which revealed that 67.1% of HEWs had RUTFs. ⁴⁵ The possible reason for this difference supported by the qualitative result revealed that shortage of RUTF because HEW did not send the request report form from the health post on time.

Compliance of HEWs with the treatment guidelines was 54%, which was poor based on the judgment parameter. This finding is lower than the study conducted in a community-based cross-sectional study in Benishangul, where 67% of cases, health extension workers were in line with the guidelines in providing nutrition services. ⁴⁶ This finding was also lower than the study conducted on the implementation fidelity of a community-based infant and young child feeding intervention in Tigray, which found that 95% of HEWs adhered to the intended protocol in providing nutrition service to the mothers. ¹⁹ The possible justification for the differences supported by qualitative findings revealed that due to the low availability of HEW who got training on national community-based nutrition guidelines for growth monitoring and promotion in the majority of health posts.

The compliance of HEW toward weight measurement was 100% performed correctly. This finding was higher than an OTP implementation study in Dehana District, Ethiopia, weight was measured in 95.2% of children in line with the protocol. ³⁰ This study differs from a study conducted in Pakistan, where weight was measured correctly in 57.4% of children. ⁴⁷ The difference might be due to the utilization of available community nutrition measurement scales.

In this evaluation, compliance of HEW with the guideline 73.3% of children’s edema status was correctly assessed. This finding was higher than in an OTP implementation study in Dehana District, Ethiopia, edema was checked for in 46.6% of children in line with the protocol. ³⁰ These findings were lower than the study conducted in a rural area of Mali, where 78.4% of children’s edema was correctly assessed. ³⁶ This finding is also lower than a cross-sectional study in Pakistan, which revealed that edema was assessed correctly for 87.5% of children. ⁴⁷ The possible difference might be because of HEWs’ commitment and training status with nutritional assessment services.

In this evaluation, compliance of HEWs with MUAC measurement was 98.12% correctly assessed. This finding was similar to an OTP implementation study in Dehana District, Ethiopia, MUAC was measured in 98% of children in line with the protocol. ³⁰ These findings have similar to the study conducted in a rural area of Mali, where mid-upper arm circumference (MUAC) was correctly assessed in 96.8% of children. ³⁶ This finding is higher than a cross-sectional study conducted in Pakistan, in which mid-upper-arm circumference was measured correctly for 60% of children. ⁴⁷ The possible difference might be due to the evaluation assessment tool and easy to perform.

From the evaluation finding, 94.37% of children’s vitamin A needs were assessed and supplementation was done based on the protocol. These findings had great differences from the study conducted in a rural area of Mali, where 33.3% of children’s vitamin A supplementation needs were correctly assessed and supplemented. ³⁶ The possible reason might be due to differences in the health care system of the country.

The compliance of HEW providing nutrition education for mothers was 40% about appropriate infant and young child feeding practices, breastfeeding, and balanced diet. This finding is higher than a cross-sectional study conducted in Pakistan, where only 4.0% of children received nutritional counseling messages. ⁴⁷ The possible difference might be case study design could get more relevant information with different observational measurement tools from cross sectionals studies.

Compliance of HEW with nutrition assessment of children by using length, WFA, and nutritional classification by WFA growth chart was 0% which was poor. This evaluation findings lower than the study in the South Wollo Zone of northeastern Ethiopia, which showed that 58.4% of the provider adhered to national guidelines on growth monitoring. ³⁷ This finding, almost congruent with a study conducted on adhesion to growth monitoring procedures by a service provider in South Africa, reveals that anthropometric measurements were performed inconsistently and the weight information was incorrectly recorded on the health charts. ⁴⁸ The possible explanation for this variation might be because of HEWs’ insufficient measuring skills to perform GMP in the service area.

Strengthen and Limitations of the Evaluation

This evaluation assessed the process of community-based nutrition service implementation in terms of resource availability and HEW compliance with the guidelines, which are important to organizational value and stakeholders’ information needs however this result was limited to western Ethiopia. The study employed a random sampling technique, which was representative. The limitation of the evaluation was the Hawthorn effect during the observation might have overestimated compliance with HEWs. To minimize this limitation, the first 2 observations in each health post were dropped, and finally, 30 non-participatory observations were taken for final analysis.

Conclusion and Recommendation

The overall process of community-based nutrition service implementation for under-2-year-old children in the Degadamot district health post was fair in terms of availability and compliance, which need urgent improvement based on predetermined judgment criteria. The availability of resources was judged to be fair. The training status of HEW and HDA was judged as fair and poor, respectively. Quick nutrition reference guidelines, growth charts, and IEC materials were judged as poor. The compliance of HEW with the national guidelines during service provision was poor in the health post. Ministry of Health and stakeholder groups to provide regular supportive supervision of HEW and ensure all health posts with trained HEWs in CBN service have been performed accordingly. to improve the ordering and distribution of supply and drug products to avoid stock-outs.