Implementation of a health management mentoring program: year-1 evaluation of its impact on health system strengthening in Zambézia Province, Mozambique

Abstract

Background: Avante Zambézia is an initiative of a Non-Governmental Organization (NGO), Friends in Global Health, LLC (FGH) and the Vanderbilt Institute for Global Health (VIGH) to provide technical assistance to the Mozambican Ministry of Health (MoH) in rural Zambézia Province. Avante Zambézia developed a district level Health Management Mentorship (HMM) program to strengthen health systems in ten of Zambézia’s 17 districts. Our objective was to preliminarily analyze changes in four domains of health system capacity after the HMM’s first year: accounting, Human Resources (HRs), Monitoring and Evaluation (M&E), and transportation management.

Methods: Quantitative metrics were developed in each domain. During district visits for weeklong, on-site mentoring, the health management mentoring teams documented each indicator as a success ratio percentage. We analyzed data using linear regressions of each indicator’s mean success ratio across all districts submitting a report over time.

Results: Of the four domains, district performance in the accounting domain was the strongest and most sustained. Linear regressions of mean monthly compliance for HR objectives indicated improvement in three of six mean success ratios. The M&E capacity domain showed the least overall improvement. The one indicator analyzed for transportation management suggested progress.

Conclusion: Our outcome evaluation demonstrates improvement in health system performance during a HMM initiative. Evaluating which elements of our mentoring program are succeeding in strengthening district level health systems is vital in preparing to transition fiscal and managerial responsibility to local authorities.

Background

The United States (U.S.) President’s Emergency Plan for AIDS Relief (PEPFAR) represents the largest commitment in history by any nation to combat a single disease (1). Its re-authorization in 2008 acknowledged that the success of priority disease programming ultimately depends on the vitality of underlying health systems (2). This heralded PEPFAR’s transition away from a vertically structured emergency response towards programming with more holistic, sector-wide approaches. Three of the five goals articulated in PEPFAR’s renewed strategy are relevant to the present work: 1) improve sustainability, 2) bolster partner governments’ capacity to lead in-country efforts, and 3) integrate disease-specific programming within existing health infrastructures (3).

Health systems should be conceptualized not only by their individual building blocks [e.g. Human Resources (HRs), financing, supply chain management, facility infrastructure], but also in terms of the dynamic interrelations between these elements (4–7). Strong systems are essential to achieving improved clinical outcomes because they are able to holistically and equitably respond to a population’s healthcare needs through coordinated efforts of people, institutions, and resources (8,9). The complex, multifaceted, and highly contextual nature of health systems poses challenges for their quantitative evaluation; as such, the World Health Organization (WHO) has increasingly emphasized the need for innovative techniques for program monitoring using system-wide approaches (10–12).

Friends in Global Health, LLC (FGH) is a Non-Governmental Organization (NGO) affiliated with the Vanderbilt Institute for Global Health (VIGH). FGH provides technical assistance to the Ministry of Health (MoH) (Ministério de Saúde, MISAU) in Zambézia Province, Mozambique through a U.S. Centers for Disease Control and Prevention (CDC) PEPFAR funded program called Avante Zambézia (“Move Forward Zambézia” in Portuguese) (13–19). Mozambique is one of the poorest nations in the world, ranking 178 of 187 countries on the 2014 United Nations Development Program’s Human Development Index (20). The country faces vast health workforce constraints, with only 2.4 doctors and 21 nurses per 100,000 persons (21). Challenges in healthcare delivery are particularly exacerbated in Zambézia Province, which had an estimated HIV prevalence of 12.6% in 2009 (22). Through a cooperative agreement with CDC, FGH has been responsible for providing technical assistance to facility-based HIV care and treatment scale-up since February 2007 in 10 of the 17 districts in Zambézia Province (Figure 1).

Figure 1 .

Map of Zambézia Province. Areas highlighted in gray represent districts supported by the Avante Zambézia project.

Avante Zambézia initiated a Health Management Mentorship (HMM) program in January 2013 for administrative personnel in ten District Health Directorates [Serviços Distritais de Saúde e Mulheres Acção Social (SDSMAS)]. The SDSMAS serve a vital role in the National Health System, and are responsible for planning programs, allocating finances, managing HRs, distributing medical supplies, and collecting data (21). However, many of these SDSMAS administrative personnel are clinicians without formal training for such managerial duties, but were brought to their current roles as a result of the dearth in HRs stemming from both external and internal brain drain across the Mozambican health sector (23).

Although significant attention has been given to the development of clinical and public health skills for frontline providers in low-income settings, evidence for strategies to enhance the management and leadership capacities needed to strengthen health systems is more limited (4,6,21). Management training has shown improvements in quality in Uganda, as well as in various high-income settings (24–26).

The FGH HMM program is uniquely designed in that it is housed within a vertically financed, HIV-specific commitment, yet striving to promote system-wide, horizontal impact. Such “diagonal” approaches within global health initiatives are relatively new, and evaluations are few from low-income settings (27–29). The goal of the FGH HMM initiative is twofold: to strengthen the district health systems through sector-wide capacity building of frontline managers, and to create a mechanism for ongoing monitoring of health system performance. These two goals overlap in that the tools for collecting these data can guide system strengthening efforts by facilitating effective and efficient decision-making by managers. This manuscript describes the initiation of the HMM program and reports a preliminary analysis after year one (January-December 2013) of its implementation by measuring quantitative changes in district health system functioning.

Background and identified challenges

Mozambique’s annual district health budgets are created from fiscal reports submitted in the fourth quarter of the prior calendar year. Once approved at the national level, the first quarter’s disbursement is made, with all subsequent disbursements dependent on the submission of financial reports from the previous quarter. Poor district compliance with MISAU accounting procedures and/or late submissions of reports can lead to delays in financial disbursements, causing gaps in funding. As a result, districts may not receive the full amount allotted to them over the course of the fiscal year, a problem noted by the FGH leadership for many of the districts within Zambézia Province.

As part of the initial HMM planning process, numerous interrelated constraints to effective HR management were identified through conversations between members of the FGH leadership and local government health workers. These constraints include insufficient quantity and rapid turnover of staff, demotivated employees from poor implementation of career development procedures, inefficient resource allocation, and inadequate funding. As demonstrated in Figure 2, these challenges are both immensely complex and intricately interrelated, meaning they must be approached holistically. These challenges are further amplified under the climate of continual decentralization of the Mozambican health sector, a process promoted in recent years by bi- and multi-lateral donors in effort to bring decision-making closer to the people. This decentralization has occurred with limited national strategies for developing the management capacity of the local public workforce to competently assume new roles and responsibilities in addition to their clinical duties (21). As such, FGH provides a stopgap solution for increasing the workforce capacity by directly hiring healthcare workers to meet the HR demands. Through negotiations with the SDSMAS, these contracted employees are to be absorbed by MISAU within its subsequent fiscal year, which includes allocating their salaries within the annual budget. As such, the HMM program directly targeted interventions to ensure these health workers were incorporated into yearly planning and that contractual procedures were in compliance to meet this target.

Figure 2 .

Identified Human Resource (HR) challenges leading to poor performance in health system management. Darker gray represents areas amenable to change through targeted interventions by the Health Management Mentorship (HMM) program.

Avante Zambézia’s Monitoring and Evaluation (M&E) capacity building interventions are directed toward the district health statistics unit. This unit is responsible for planning district activities based on nationally set targets, ensuring the submission of monthly health indicator reports to MISAU, and overseeing stock management of all paper-based patient documentation forms and data collection registries used at the various service delivery points. The FGH leadership team noticed frequent stock-outs of these forms and registries at project start-up, leading to missing data and delays in reporting. One of the most significant constraints within this sector included the staff’s reactionary nature to daily urgencies rather than anticipating priorities through advanced planning and the use of complete data.

Transportation is a considerable barrier to the efficient functioning of the Zambézia Province health system, due in part to the limited number of vehicles available to each district, vast distances required for service delivery, and poor road conditions. The FGH leadership team anecdotally noted that inappropriate utilization of existing fleets was common, such as borrowing the district ambulance to transport health personnel to meetings outside of the district, temporarily leaving certain locations without ambulatory support. Furthermore, funds to purchase insurance and service existing fleets are limited. On average, each district has two ambulances, two or three hard-body pick-up trucks, and five to ten motorcycles; yet, at any given time, up to half of these vehicles are in disrepair and nonfunctional, resulting in dependence on NGO partners for service delivery. At project start-up, documentation on the status and use of the district health sector vehicles was poor, creating difficulties in activity planning, budget forecasting for fuel and maintenance, and justifying current and future transportation needs.

At program start-up, HMM mentors worked with provincial health authorities to identify eight domains for intervention: accounting, HRs, supply chain management, durable goods management (i.e. fixed capital), district health service administration, transportation, M&E capacity, and supervision. To support this program, FGH signed sub-grants with each district in order to directly transfer funds to the SDSMAS, complementing the existing MISAU budget. For greater transparency and ease of auditing, these sub-grant funds were transferred into district health accounts dedicated for their sole use, and defined budgetary line items and accounting tools corresponded with existing MISAU accounting practices in an effort to avoid parallel systems. FGH sub-grants amounted to approximately 30% of each district’s monthly operating budget, with the remainder of the funds coming directly from MISAU.

Methods

Program description

This paper assesses changes in four domains of system capacities after one year of the HMM program: accounting, HRs, M&E, and transportation management. The overall assumption underlying this program is that improvements in holistic health system functioning will lead to better patient outcomes. Mentoring support was provided through three regional teams; each team consisted of a leader with formal management training, a certified accountant, and a representative of the Provincial Health Directorate. Due to financial limitations in the number of staff who could be hired, as well as the size of  Zambézia Province and subsequent lengthy travel times between districts, utilizing three separate mentoring teams allowed each team to be regionally focused within the Province, providing support with the personnel available. Each team was responsible for oversight of three or four districts. Teleconferencing or other forms of virtual learning were not available due to resource and infrastructure constraints, so each team provided in-person visits to districts for one week every four to eight weeks. These mentors worked with the district level administrators for HR issues, accountants for funding issues, M&E managers for data collection issues, and district fleet managers for issues related to transport. By spending time with the managers in their own work environments and assisting them throughout day-to-day challenges, this site-based mentorship approach provided contextualized guidance and avoided sending staff to costly, off-site workshops, which cause significant disruptions in local service provision (30–32). Additionally, when not in a particular district, mentoring teams were available by phone and/or email to the district staff for troubleshooting and ongoing support. Elements of the HMM program and procedures for data collection underwent an iterative rollout process from January to March 2013, with systematic data collection beginning on a monthly basis in April 2013.

Data collection for health system indicators

Within the eight health system domains, Avante Zambézia developed 63 quantitative indicators to assess health system strength as part of the routine function of the HMM program. This analysis consolidates data collected by the mentors during their district visits on 13 of the most salient metrics within the four domains with the most data available—accounting, HRs, M&E capacity, and transportation management—in order to assess the overall strength of those sectors and the effectiveness of the HMM initiative in general (Table 1). During each visit to a district, the mentoring team documented both the possible number of times and the successful number of times the managers addressed a particular objective, providing a percentage success ratio. Each of these indicators could be measured objectively; for example, the fourth HR indicator captures the percentage of salaries that are correctly processed and paid on time. The mentoring team would measure this by reviewing the disbursement records and bank statements from the previous month to determine how many salaries out of the total number of salaries were paid on time.

Table 1 . Indicators of health system strength across four health system domains .

Accounting

1. Prepare high-quality quarterly financial reports

2. Allocate budgets to cover expenses in requisitions

3. Earmark payments based on appropriate documentation

HR

1. Comply with annual plan for absorbing NGO-hired staff into NHS

2. Allocate budget to cover costs of newly absorbed government personnel

3. Comply with annual plan for changes in staff positions (promotions, career development, etc.)

4. Allocate budget to cover salaries affected by changed staff positions

5. Process and pay all salaries by 15th of the month

6. Conduct performance evaluations for each employee on a quarterly basis

M&E capacity

1. Use government-issued guidelines when developing district plans

2. Submit and archive monthly MoH performance reports

3. Sufficient stock of data collection forms and registers based on identified demand

Transportation management

1. Report transportation use for each vehicle in the logbook

NGO= Non-Governmental Organization; M&E= Monitoring and Evaluation; NHS= National Health System; MoH= Ministry of Health; HR= Human Resources.

Data were collected using Microsoft Excel version 14, and transferred to REDCap (Research Electronic Data Capture), a secure, web-based tool hosted at Vanderbilt University (33). Data were exported to Stata version 12 (College Station, TX, USA) and Microsoft Excel for analysis. Percentage success ratios were disaggregated by district, helping to troubleshoot and initiate quality improvement on an individual district basis. Average frequencies were generated for indicators measured on a quarterly or year-end basis. For metrics captured monthly, linear regressions representing the mean of all districts that submitted a report for that particular month were run over the nine-month period to evaluate the impact of the HMM initiative (Table 2). Positive, significant (P < 0.05) correlation coefficients suggest improving trends, with larger coefficients representing greater improvements.

Table 2 . Simple linear regression analysis of performance indicators over time in rural Mozambique .

	β	R 2	Standard error	P value
Accounting domain
Allocate budgets to cover expenses in requisitions	9.16	0.57	1.24	<0.001
Earmark payments based on appropriate documentation	4.54	0.39	0.89	<0.001
HR domain
Allocate budget to cover costs of newly absorbed government personnel	9.96	0.75	0.90	<0.001
Comply with annual plan for changes in staff positions (promotions, career development, etc.)	3.15	0.22	0.93	0.002
Allocate budget to cover salaries affected by changed staff positions	8.58	0.47	1.40	<0.001
Process and pay all salaries by 15th of the month	-0.37	0.00	1.18	0.750
M&E capacity domain
Use government-issued guidelines when developing district plans	-0.58	0.28	0.14	<0.001
Submit and archive monthly MoH performance reports	1.04	0.01	1.43	0.471
Sufficient stock of data collection forms and registers based on identified demand	6.31	0.65	6.31	<0.001
Transportation management domain
Report transportation use for each vehicle in the logbook	2.50	0.18	0.82	0.004

M&E= Monitoring and Evaluation; HR= Human Resource; MoH= Ministry of Health.

Results

Through 2013, four to seven weeklong mentoring visits were made per district, a total of 62 visits across ten districts, with forms submitted from 44 (71%) of the visits. Seven of the ten districts (with the exception of Ile, Inhassunge, and Mopeia) completed year-end reports in December 2013. Initially, a schedule of visits was created so that each district would receive two visits per quarter. Due to competing priorities, transport availability, and the need to adapt the schedule to emergent needs at a district level, the mentoring teams were not able to precisely follow the schedule as designed originally.

Accounting

With the introduction of direct financing to the SDSMAS via sub-grants from FGH, the HMM program directed substantial time and energy to providing experiential training in quality accounting practices, a service previously not available. The objective was to ensure compliance of district managers with expenses and justifications for both FGH sub-grant accounts and for funds allocated from MISAU. While mentoring activities focused on all district health revenue streams, this analysis is limited to evaluation of the accounting practices of the FGH sub-grant accounts.

High quality was defined as being submitted on time and free from errors. Evaluation of Accounting Objective 1 (Preparation of high-quality quarterly financial reports) revealed a marked second quarter improvement in the proportion of financial reports achieving high-quality status, though some slippage was noted in the third and fourth quarter quality assessments. High quality was noted in 75.0% of quarter one (Q1) baseline reports, 96.7% in Q2, 89.2% in Q3, and 91.7% in Q4 (average of 92.5% for Q2-4). Of the reports returned to districts due to justification irregularities, all were subsequently resubmitted to FGH within a one-month period with corrections. At the end of year, all districts had appropriately justified and spent 100% of FGH sub-grant funds. As demonstrated in Figure 3 , simple linear regressions of mean monthly compliance for Accounting Objectives 2 (Allocate budgets to cover expenses in requisitions) and 3 (Earmark payments based on appropriate documentation) suggest improvement in their mean success ratios over time. Outlier points for the period between April and July reflect inconsistencies in documentation quality and timeliness at the district level due to the same the justification irregularities mentioned above. As the year progressed and districts became more accustomed to regular mentoring visits, these inconsistencies diminished Figure 3 ).

Figure 3 .

Evaluation of Accounting Objectives 2 (percent of budgets correctly allocated to cover expenses) and 3 (percent of payments allocated based on appropriate documentation). A) Percent of budgets correctly allocated to cover expenses (Accounting Objective 2). Simple regression coefficient (β) of 9.16 (P< 0.001). B) Percent of payments allocated based on appropriate documentation (Accounting Objective 3). Simple regression coefficient (β) of 4.54 (P< 0.001).

Human Resources (HRs)

The benchmark for HR Objective 1 (Comply with annual plan for absorbing NGO-hired staff into National Health System) was set to 90% compliance, of which only four of the ten (40%) districts met this target (Figure 4). Simple linear regressions of mean monthly compliance for HR Objectives 2 (Allocate budget to cover costs of newly absorbed government personnel), 3 (Comply with annual plan for changes in staff positions), and 4 (Allocate budget to cover salaries affected by changed staff positions) suggested improvements over time in the mean success ratios of district budget allocations for hiring NGO contracted staff, as well as in the allocation of budgets to cover the salary increases resulting from implementation of promotions and other career development planning. In the month of August, the provincial head of the HR department was on vacation, which inadvertently led to a backlog at the district level in the ability to process new staff positions. The consequences of this scenario are reflected as outliers in each objective for that month (Figure 5). The coefficient for the linear regression (β) of HR Objective 5 (Payment of staff salaries by the 15th of the month) showed a slight decline, but this value was not statistically significant. Evaluation of HR Objective 6 revealed a marked and steady deterioration in the proportion of completed employee performance evaluations over time. Performance evaluations were completed in 70.3% of Q1 baseline reports, but only 48.3% in Q2, 34.6% in Q3, and only 8.5% in Q4 (average of 30.5% for Q2-4).

Figure 4 .

Success ratio of Human Resource (HR) Objective 1: Percent compliance with annual plan for absorbing NGO-hired staff by National Health System per district, at year-end. Values greater than 100% represent staff who were hired and absorbed by the National Health System above what was planned at the beginning of the year.

Figure 5 .

Human Resource (HR) Objectives 2, 3, and 4. A) Percent of times budget correctly allocated to cover costs of new personnel (HR Objective 2). Simple linear regression coefficients (β) of 9.96 (P < 0.001). B) Percent compliance with annual plan for changes in staff positions (HR Objective 3). Simple linear regression coefficients (β) of 3.15 (P = 0.002). C) Percent of times budget correctly allocated for salaries affected by changed staff positions (HR Objective 4). Simple linear regression coefficients (β) of 8.58 (P < 0.001).

Monitoring and Evaluation (M&E) capacity

The M&E capacity domain showed some deterioration, as summarized by simple linear regression coefficients (β) of mean monthly compliance for M&E Objective 1 (Use of government-issued guidelines when developing district plans) over time (β = -0.58, P < 0.001). Furthermore, M&E Objective 2 (Submit and archive monthly MISAU health indicator reports) showed no significant change over the first year of analysis. In contrast, Objective 3 (Sufficient stock of data collection forms and registers) suggested improvement over time (β = 6.31, P < 0.001; Table 2).

Transportation management

The HMM program targeted district transportation managers for training in fleet administration and supervision, which included developing weekly vehicle usage plans, creating schedules for routine maintenance, and utilizing a vehicle mileage and fuel consumption logbook. Linear regression for Transportation Objective 1 (Transportation logbook used and updated for each health service vehicle) suggested improvement in the mean success ratio over time (β = 2.50; P = 0.004).

Discussion

This outcome evaluation assessed the effect of a HMM initiative on overall health system functioning within four domains. Our experience reflects the realities of severe limitations in one of the world’s most resource-constrained nations (34). In this context, health worker shortages and weak health systems threaten to undermine the MISAU’s ability to attain key health goals (35). Lessons learned during this initial year of implementation will foster continuous strengthening of the health systems in these ten districts. Table 3 highlights the overall successes and challenges of the HMM program by domain as well as areas for future work.

Table 3 . Health management mentoring: successes, challenges, and future work .

Overall

Successes

• Early engagement with health authorities on program design and defining domains for targeted intervention

• Implementation of district mentoring visits jointly between FGH and Provincial Health Directorate representatives • Improved overall coordination and planning between FGH and the Provincial and District Health Directorates

Challenges

• Constant evolution of MISAU and donor priorities causing reactionary changes in health services demands, even after annual plans established

• Incomplete decentralization process to the district level with inconsistencies in decision-making between provincial and district health authorities

• Ever increasing demand for expansion of HIV and other programs to the periphery, introducing further strains on both district health and NGO staff to ensure quality service delivery and supervision

• Incongruent planning and fiscal years between MISAU and FGH

Accounting domain

Successes

• Improved accounting practices resulting in fewer delays in health sector budget disbursements

• Greater district health spending power to meet needs due to fewer delays in government disbursements, complimented with infusion of funds from FGH sub-grants

• Greater transparency of district spending from all funding streams resulting from frequent accounting audits of the HMM program

Challenges

• Lack of district finance/accounting personnel with appropriate training in the area

• Greater fiscal demands on health sector as services get pushed further into the periphery

• Ongoing difficulties in forecasting and budgeting for needs of periphery due to insufficient staff and transport to visit on a regular basis

HR domain

Successes

• More rapid increasing of health sector workforce through coordinated efforts in which FGH provides stop-gap initial labor contract and financing, while preparations and budgets found for absorption within the national health system

• Larger number of district workforce with advancing career development

• Less frequent delays in payment of monthly salaries

Challenges

Continued high staff turn over

• Continued high level of staff out of work site due to increasing demand for training

• Lack of evidence-based results as to whether the above HR successes contribute improved staff work satisfaction and retention

M&E domain

Successes

• Improved stock management of patient forms and registries, with improved forecasting of needs including peripheral clinics

Challenges

• Difficulties in maintaining improvements related to district activity planning based on government-issued guidelines, due to ever evolving changes in program targets and priorities

Transport management domain

Successes

• Less time of existing fleets out of service due to improved routine maintenance schedule

• Better documentation of vehicle usage

Challenges

• Insufficient quantity of vehicles to meet district demand

• Aging fleet in poor conditions with slow and unpredictable replacement with new vehicles Continued extreme road conditions and distances causing excessive wear and tear on existing fleet

Future work

• Beginning 2014, regional mentoring team approach transitioned to single team per each of ten districts, in order to provide more frequent support and respond to the increasing needs caused by further expansion to the periphery

• Continue to further reinforce indicators in existing domains

• Implementation of impact evaluations to determine effects of health system strengthening on service delivery and patient outcomes

FGH= Friends in Global Health; MISAU= Ministério de Saúde; HMM= Health Management Mentorship; NGO= Non-Governmental Organization; M&E= Monitoring and Evaluation; HR= Human Resource.

One important key to success in this quality improvement effort was FGH’s open and collaborative relationship with provincial and district health authorities. Leaders from MISAU were integrally involved in the design of program activities, which encouraged mutual trust. In following best practice guidelines in global health by involving local stakeholders, our interventions were designed to target areas of joint priority and need, a practice encouraging both relevance and sustainability of the program. This is particularly important as PEPFAR seeks to fully transition authority to local entities, a process we recognize can have precarious consequences if done too quickly or too cheaply (29).

A strength of this study is its internal validity, as trained members of the mentoring team objectively and consistently captured the data, though no formal evaluation as to the consistency and accuracy of the reports was collected. The mentoring team was not incentivized in any way for improved district performance, so we do not believe that a desire for the program’s success reduced the validity of the measurement. However, because this program was highly contextualized to Zambézia Province through involvement with the MISAU, the applicability of these results to other countries or regions of Mozambique may be limited. Furthermore, this program required a high investment of human capital, as there were three mentoring teams of three mentors each who traveled frequently to individual districts for weeklong instructional visits. This model was time consuming and costly, and we recognize that our results might not be easily replicable in other settings where it is less feasible for mentors and/or mentoring teams to provide such frequent and contextualized instruction.

This study is limited in that our monthly data stem from districts that submitted a report for that particular month. Any omission of data from non-reporting districts is likely to skew the results toward more positive findings. Additionally, our data are limited in that we did not include a control group for the mentoring initiative, i.e. all districts participated in the program, and pre-intervention baseline data were not collected. As such, the changes in these outcomes are measured exclusively against the duration of the intervention, making it impossible to distinguish progress from possible secular trends. However, since some elements improved and others did not, we believe there are important insights regarding which interventions might have been working better than others.

Of the four domains, district performance in the accounting domain exhibited the strongest and most sustained improvements. The sub-grant funds supplemented the existing MISAU budget, and the budgetary line items and accounting practices for these additional funds were streamlined to be consistent with the original budget, which enabled greater financial transparency and ease of auditing. Mentors worked directly with FGH accountants to develop, maintain, and strengthen the quality of the project finances.

Mozambique’s limited health sector workforce is one of its greatest barriers to the independent provision of quality services. While the issue of inadequate number of health workers is beyond the control of local authorities, the HMM program concentrated its efforts on improving simple areas which can have a salutary impact. For example, receiving salaries on time and understanding opportunities for improvement and/or promotion are vital to worker satisfaction and retention. In addition, the overall functioning of the health system is currently contingent on the partnership between the Provincial Health Directorate and FGH for hiring health workers, a model which helps to ensure that the provincial and district HR departments are prepared to integrate new health workers as they become available (Figure 4).

District HR management saw improvements in its ability to pay salaries on time, initiate procedures for health worker career development, and plan and budget for new personnel (Figure 5). However, evaluation of HR Objective 6 revealed a marked decline in the proportion of completed employee performance evaluations over time. Procedures for advancing career development remain weak and slow throughout the provincial HR sector, though improvements have begun. We acknowledge that our efforts in this program’s first year have not consistently improved HR indicators, but we believe these issues are now garnering much-needed attention, which we hope will initiate their progress and continued reinforcement.

The M&E capacity domain demonstrated weak progress across year-one. Extensive annual planning and preparations occurred at the beginning of the year in response to HMM encouragement; however, as the year progressed, urgent demands on the health sector frequently required action outside of these original plans. Our on-site mentoring model aimed to diminish staff time away from the district, yet the M&E staff were frequently absent from their worksites for other mandatory national meetings and trainings. These staff were often not available on the week of the HMM visit, yet our interventional was fundamentally designed for in-person contact. As a result, we are exploring ways to revise the program model to be suitable for remote mentorship, particularly if M&E staff will continue to be off-site on a frequent basis.

Activities are ongoing to promote data-driven decision-making to enhance the quality, accuracy, and consistency of data collected. Fundamental to a district’s ability to do this is the establishment of procedures to ensure monthly health sector plans (e.g. campaigns for vaccination, supervision visits, etc.) align with national priorities and guidelines. Additionally, resources should be allocated at service delivery points to improve the quality of data collected.

Conclusion

A strategy to deploy teams of mentors to build capacity by equipping managers with skills in planning, priority setting, and problem solving across eight target health management domains was implemented with some early success. Intervention areas were defined based on assumptions that improvements in overall system performance will result in better health outcomes for patients. We are beginning to see signs of progress and sustainable reinforcement of health systems as a result of the HMM program, and learned important lessons for moving forward with this program. However, it is important to keep in mind that such improvements are nested within a complex array of challenges that Zambézia Province must overcome in order to be ready for full transition of financial and administrative authority of PEPFAR activities. Once quality improvements are more consistent and sustained across all objectives, we plan to undertake an impact-level evaluation that incorporates an analysis of costs and benefits, including an exploration of the relationship between HMM indicators and metrics capturing patient-related  outcomes.

Ethical issues

The study observed trends that did not include any identifiers or information on human subjects, and was approved by the Zambézia Inter-institutional Committee for Bioethics in Health and the Institutional Review Board of Vanderbilt University Medical Center, Nashville, TN, USA.

Competing interests

Authors declare that they have no competing interests.

Authors’ contributions

TDM, AM, MN, AC, LS, and JM were involved in the design of the HMM program and data collection. LJE and TDM were involved in the design of the study. LJE, TDM, CWW, and SHV were involved in data analysis and drafting the initial manuscript. All authors contributed equally to edits and reviews of subsequent drafts of the manuscript.

Authors’ affiliations

¹Vanderbilt Institute for Global Health, Vanderbilt University, Nashville, TN, USA. ²Friends in Global Health, Maputo, Mozambique. ³Vanderbilt Institute for Global Health, Department of Medicine, Division of Infectious Diseases, Vanderbilt University, Nashville, TN, USA. ⁴Vanderbilt Institute for Global Health, Department of Pediatrics, Division of Infectious Diseases, Vanderbilt University, Nashville, TN, USA.

Key messages

Implications for policy makers

• Health worker shortages and weak health systems threaten to undermine the ability to attain key health goals.

• A strategy to deploy teams of mentors to build capacity by equipping district health managers with skills in planning, priority setting, and problem solving across eight target health management domains was implemented with some early success.

• Mutual trust stemming from an open and collaborative relationship with provincial and district health authorities was key to program success.

• In following best practice guidelines in global health by involving local stakeholders, our interventions were designed to target areas of joint priority and need, which encourages both relevance and sustainability of the program. This is particularly important as President’s Emergency Plan for AIDS Relief (PEPFAR) seeks to fully transition authority to local entities, though we recognize that such a process can have precarious consequences if done too quickly or too cheaply.

Implications for public

A health management mentoring initiative was implemented as part of a larger President’s Emergency Plan for AIDS Relief-funded (PEPFAR-funded) HIV care and treatment scale-up program, based on assumptions that improvements in overall system performance will eventually result in better clinical outcomes for patients. Our results show that, in Zambézia Province, Mozambique, mentoring specifically targeted towards the development of health management skills was correlated with improvements in accounting, Human Resources (HRs), Monitoring and Evaluation (M&E) capacity, and transportation management when evaluated at the end of year one. In order to strengthen the overall functioning of health systems in resource-limited settings, it is necessary to equip healthcare managers with the knowledge and skills needed for them to be effective.

Citation: Edwards LJ, Moisés A, Nzaramba M, Cassimo A, Silva L, Mauricio J, et al. Implementation of a health management mentoring program: year-1 evaluation of its impact on health system strengthening in Zambézia Province, Mozambique. Int J Health Policy Manag 2015; 4: 353–361. doi: 10.15171/ijhpm.2015.58