Table 1.

Core indicators used to develop the BSC for HEP in rural Ethiopia

Indicators	Description and the factors that went into the indicator
Capacity for service provision
HEP staffing	This deals with the number and qualification of service providers at the village level (including HEWs and volunteer health promoters). The standard for HEP staffing at the village level is to have at least two HEWs and one health promoter for every 40 households
Health facility infrastructure	The infrastructure index deals with whether the health post has the required infrastructure, facilities, and amenities that enable it to render services as per the standard. It covers the availability of health post building with at least three rooms, the availability of electricity, water and telephone communication, toilet access to patients, and availability of means of transportation
Equipment functionality	It measures whether the health post is equipped with the minimum medical equipment required to deliver the HEP services based on the standard. The availability and functionality of the following were considered: examination bed, thermometer, blood pressure apparatus, stethoscope, delivery bed, home delivery kit, neonatal resuscitation mask and bag, baby weighing scale, Salter scale, measuring board, and refrigerator
Commodity availability	This covers the availability of essential medical supplies and vaccines required to deliver the basic health services. The following medical supplies were considered: Coartem, oral rehydration salt, contraceptives, ergometrine, iron tablets, folic acid, vitamin A, amoxicillin, RDT for malaria, HIV test kit, and analgesics. The various antigens required for child and mother immunizations were also included
Supply management system	This indicator deals with the supply, storage, distribution, and management of medical supplies at the health post, health center, and woreda health office. At the health post, the frequency and waiting time of supply replenishment, the adequacy of supplies, and the ability of HEWs to identify expired drugs were considered. At the health center, availability of adequate stocks of medical supplies (drugs, vaccines, medical equipment, and supplies) for distribution to health posts and availability of adequate and secure storage facility were considered. The availability of medical supplies for all health posts and their sources were considered at the district level
HEP guideline and IEC use	It deals with the availability of IEC such as posters and flip charts covering the various HEP service packages and summary charts, and whether the materials were displayed plainly at the health post. It also covers the availability of guidelines for HEP implementation, obstetric care, growth monitoring, and malaria case management
HMIS use	The HMIS use indicator deals with the availability and proper use of registers for the various HEP service packages including birth and death registration. It also covers reporting and use of information for decision making
Health center capacity	This indicator covers factors that affect the readiness of the health center to serve as referral facility for the health posts. The factors considered included the distance from the health post, availability of service for 24 hours, availability and use of injectable drugs and supplies for emergency obstetric care (such as diazepam, magnesium sulfate, oxytocin, ergometrine, amoxicillin, gentamicin, intravenous infusion, incubator, and forceps and vacuum extractor for assisted delivery), availability of staff who can administer the drugs and perform the procedures, availability of blood transfusion and cesarean section services, and availability of other services that require referral
Management capacity	This indicator deals with human resource performance management, planning process, HEP performance monitoring, and partner collaboration. The human resource performance management considered the availability and approaches used to conduct performance assessment of HEWs, ensure adherence to code of conduct, and motivate good performing HEWs. The planning process covered the use of baseline information for planning and the involvement of HEWs in planning process. The HEP performance monitoring covered the involvement of health centers and district health offices, the performance monitoring methods used (such as review of monthly reports, and regular meetings with HEWs), the use of information for decision making, and the comprehensiveness of the indicators used for HEP performance monitoring. The partner collaboration covered the level of support to health posts from district and kebele administrations
Supervision capacity	The supervision index covers the human resource capacity for supervision available at the health centers and district health offices, the logistics (supervision guidelines and checklists, transportation and budget for supervision), availability of supervision plan and schedule, performance in supervision (relative to plan and the number of supervisory visits made to health posts over 3 months), the use of appropriate supervision methods and quality of supervision (which is characterized by multidisciplinary team, frequent and regular, person-to-person discussions, discussions with local community and leaders, useful and supportive guidance, and supervisors listen and value HEWs' suggestions), and provision of feedback and follow-up to ensure that correction measures are taken
Financial systems
Financing referral system	The financing referral system covers the availability of sustainable arrangements for financing emergency referral services (such as funds/fuel set aside, community insurance scheme, or revolving funds) and availability of barriers for referral system (such as lack or high cost of transportation and cost of services and drugs at referral health facilities)
Financing medical supplies	The financing medical supplies cover the availability of adequate funding for medical supplies and whether patients are referred due to lack of basic medical supplies
Human resources
HEW knowledge	The HEW knowledge index covers HEW's competence in providing the key HEP service packages. The assessment included knowledge on maternal health (such as on benefits of ANC, signs and monitoring of labor, immediate newborn care, and management (diagnosis and actions taken by HEWs) when a woman comes with heavy bleeding during or after delivery, retained placenta, obstructed labor, infection, eclampsia, severe anemia, and severe malaria); neonatal and child health (including on diagnosis and action taken by HEWs when there is neonatal infection and underweight, acute respiratory infection, diarrhea, and danger signs); family planning (counseling skills regarding the effectiveness, benefits, and risks of all available methods); malaria diagnosis and treatment; and immunization administration schedules
In-service training	This indicator covers the provision of in-service training to HEWs on various HEP service packages in the year before the survey and provision of in-service training to HEW supervisors on HEP and supervision techniques
HEW satisfaction	This indicator covers the level of HEW satisfaction on salary and benefits, management handling, opportunity with professional development, support and relationship with key stakeholders, and housing condition
HEW perception	This indicator deals with perceptions of the HEWs on job responsibilities (including workload and difficulty relative to level of training), remuneration (fairness relative to workload, level of training and workers in other sectors), management (relationship with managers and supervisors and impact of managers on benefits), constraints affecting job performance, social and organizational obstacles, and living conditions
Retention of HEWs	It deals with retention of HEWs taking into consideration the number deployed and the number left since HEP started in the district, which was used to generate annual retention rate
Service delivery
Service availability	This indicator deals with whether all the HEP service packages are render at the health post
Quality of HEP service	This indicator covers the quality of service provision on key aspects of HEP, including cold chain management system that ensures potency of antigens, and important aspects of newborn care services such as handling of umbilical cord, practicing suction of newborn, weighting, drying and wrapping of newborn, and administration of OPV and BCG vaccine at birth
Infection prevention	The infection prevention indicator covers availability of infection prevention practices (availability and use of syringes, gloves, and antiseptics), handwashing facility (access to water and soap for patients and availability of handwashing apparatus with tap at the health post), and proper waste disposal (including availability of dust bins and medical waste disposal mechanisms such as pits, burning, or incineration)
Referral linkage	The referral linkage index deals with health post, referral health facility, and clients' aspect of the referral system. It considered the type of referral facility, distance, availability of communication and means of transportation, reason for referral, availability of referral forms, functional two-way linkage (referral and feedback), willingness of patients to go to referral health facility, and availability of factors preventing patients from going to referral facility and affecting the referral system (such as poor customer and care services, long waiting time and high cost of service at the referral facility, lack of awareness, distance, and poor road)
HEW time use	HEW time use indicator deals with whether HEWs use their time as per the HEP standard. It considered the number of days the health post is opened per week, whether the health post is opened on weekends, number of hours HEWs work daily, and time spent at the health post vs. community level (on average 50% of time for each level). Availability and use of standard manual on apportioning their time among the 16 HEP service packages was also included
Model family package delivery	The model family package delivery indicator covers the quality of implementation (relative to the standard procedure) and level of household coverage. The quality of implementation considered the knowledge of HEWs on the standard procedures, actual implementation procedure (number of households enrolled per round, number of hours households were trained before graduation and number of months took to complete the training), training topic selection procedure (priority is given to areas that are easily implementable, do not create conflict with culture, and showed success previously), use of demonstration units for training, process and criteria to determine eligibility for graduation, involvement of key stakeholders (health promoters, kebele council, and HEW supervisor) in training and determining eligibility, and ceremony to celebrate graduation and provide awards. The level of coverage considered number of households graduated assessed relative to the total households in the kebele
Community engagement
Awareness and access	The community awareness and access to HEP indicator considered the following factors: awareness about HEP, whether the individual visited HEW in the previous month, whether it was difficult to get to the health post or the HEW and had to wait too long before receiving care during the last visit, and availability of information on the schedule of HEW displayed outside the health post
Community and health  promoters participation	The community and promoter participation indicator considered the level of participation of various sections of the community in the implementation of HEP as per the standard. The participation of health promoters covered the following factors: number of households assigned, availability of work plan and service registers, whether they prepare monthly reports and participate in monthly review meetings, their level of satisfaction with their role, and the level of support they receive from key partners. It also considered participation of elders, associations, religious and kebele leaders, health promoters, and the community at large in preparing work plan and implementation of HEP
Model family engagement	The model family engagement indicator deals with the level of participation of the community in attending training and supporting training of other households (including encouragement and provision of training). It also covered the level of happiness of becoming model family and their interest to serve the community
Village health committee	The village health committee functionality indicator covered the availability of health committee, its involvement in planning and implementation and its participation in organizing emergency transportation of referral cases
Delivery output and outcome
Maternal and child health output	This indicator deals with the number of key maternal, neonatal, and child health services provided at the health post in the year before the survey assessed relative to the respective population and annual target set by the FMOH. The services included ANC, assisted delivery, immunization, PNC, cases managed (diarrhea, acute respiratory infection, and malaria), nutrition and growth monitoring, and HIV/AIDS education services
Family health output	This indicator deals with the number of family planning clients at the health post in the year before the survey assessed relative to the target set by FMOH
Satisfaction with services	This indicator measures the level of community satisfaction with the services, specifically on the availability and quality of the various HEP services packages. It covers the satisfaction rate on key maternal and child health services, curative services, sanitation, health education, and model family implementation. It also considered whether the community members would visit again the health post or recommend the same health post to others, and their satisfaction with cleanliness of the health post environment
Satisfaction with HEW	This indicator deals with the level of community satisfaction on HEW's technical skills (such as explaining in understandable manner, making helpful suggestions, and discussing treatment options), communication skills (such as caring, friendly, complete explanation, attentive, and respectful) and social skills (such as participation in community events, use local language, good conduct/ethics, and confidentiality of personal information)
Satisfaction with HEP	The overall satisfaction with HEP indicator covers overall community satisfaction levels with respect to addressing the health needs of households and the community at large and their level of happiness with the availability of HEP services in their village

AIDS = acquired immunodeficiency syndrome; ANC = antenatal care; BCG = Bacillus Calmette–Guérin; BSC = balanced scorecard; FMOH = the Federal Ministry of Health; HEP = Health Extension Program; HEW = Health Extension Worker; HMIS = health management information system; HIV = human immunodeficiency virus; IEC = information, education and communication; OPV = oral polio vaccine; PNC = postnatal care; RDT = rapid diagnostic test.