Health System Governance Evaluation: A Scoping Review

Abstract

Background:

Governance is one of the critical functions of the health system. Good governance of the health system leads to better performance and outcomes. Evaluation is the first step to improving health system governance. Therefore, this research aimed to identify evaluation tools for health system governance.

Methods:

In the current scoping review, we searched all documents related to health system governance evaluation indexed in Medline, EMBASE, ProQuest, Scopus, Cochrane Library, Science Direct databases, and Google Scholar search engines to 2022, extracted, and assessed. Finally, documents were selected and analyzed by thematic analysis.

Results:

Thirty tools were found to evaluate health system governance. Among the proposed tools, 11 specific tools have been designed just for health system governance evaluation, while others have governance as a component of health system evaluation. Health system governance’s significant components are health policy-making, strategic planning, organizing, stewardship, and control. Indicators such as accountability, participation, transparency, equity, efficiency, accountability, corruption control, effectiveness, ethic, the rule of law, and sustainability could be used to evaluate the health system governance.

Conclusion:

Different tools have been used to evaluate governance worldwide, and each governance evaluation tool has different components suitable for governance evaluation. However, these tools cannot fully evaluate governance and have shortcomings. A comprehensive evaluation of governance and sharing lessons denrael will affect the health system's capacity and ability to provide quality, safe and effective health services. It will lead to the stability of the health system.

Introduction

Health and well-being are shaped by multiple determinants that span political, cultural, social, economic, environmental, health systems, and commercial sectors and domains. Addressing these wider determinants and making a positive difference in health and well-being outcomes requires governance and policies that are intersectoral and multidimensional (1). An increasingly globalized, developed, interconnected, and educated world has provided a rapidly changing global, socioeconomic, technological, environmental, demographic, and health. Demographic shifts, the epidemiological transition of disease, the increasing frequency of outbreaks, epidemics, and pandemics, as well as the increasing politicization of health, have created an urgent need to address the new challenges presented in this context (2). These challenges can be addressed through strengthened governance, with health and well-being at its center. Multi- and intersectoral governance and synergized approaches to governance system levels, sectors, institutions, communities, cities, and countries are essential for navigating this changing and evolving context to achieve sustainable and equal health and well-being outcomes for all (3).

Governance was introduced into the health care literature following the World Health Report 2000 by the WHO. In this report, WHO referred to stewardship as one of the main functions of the health system, along with financing, resource generation, and service provision, and defined it as “the careful and responsible management of the welfare of the population” and the essence of good government (4). This function has expanded over time, and the WHO replaced stewardship with the phrase ‘leadership and governance in a 2007 report (5). The governance function involves “ensuring that strategic policymaking is combined with effective oversight, coalition building, regulation, attention to system design, and accountability” (5).

Good governance in the context of health refers to the “formulation and implementation of appropriate policies and procedures that ensure effective, efficient and ethical management of all aspects of health systems, in a manner that is transparent, accountable, follow the rule of law and minimizes corruption” (6). Ministry(s) of health/health authorities and other relevant bodies could utilize this definition in general.

There has been an increasing interest in evaluating health system governance because of the characteristics of the health sector, such as asymmetry of information and the growing influence of health system stakeholders (7). Governance affects other functions of the health system, and good governance plays a crucial role in improving the system’s performance and, ultimately, health outcomes (8, 9). Another reason for increasing interest in governance in the health sector is the 2006 report by Transparency International. It discusses corruption in the health sector as a global problem that threatens good governance in both high- and low-income countries (10). Any impairment of the health system’s governance function can hinder policy implementation. Therefore, evaluation is the first step to improving governance in the health sector and prepares the ground for implementing different governance strategies. The WHO has also emphasized governance evaluation. Evaluation is essential to good governance and a key requirement for improving governance in the health sector (11).

The assessment of governance should facilitate policymakers’ understanding of the complex and multidimensional concept in the absence of a merged definition. Recognizing the various unclear and abstract definitions of governance, most international organizations have adopted a strongly normative approach to understanding this phenomenon, suggesting lists of principles making up good governance (4, 12, 13). However, agreeing on core principles of good governance has resulted in applying many lists of principles by different organizations.

The normative literature has proceeded by suggesting lists of principles and characteristics that make up good governance, and particular concerns supplement these for health system governance (4, 12–14). There are overlaps on these lists, and they are variably supported by empirical studies of governance in practice. The most relevant principles/domains related to Health System Governance are participation, transparency, accountability, the use of information, responsiveness, ethics, equity, efficiency and effectiveness, and the rule of law, and strategic vision (15). This set of principles was based on the internationally recognized United Nations Development Program (UNDP) principles of good governance (12) and is one of the most comprehensive definitions (3).

We aimed to identify existing evaluation tools for health system governance.

Methods

This study used the scoping review method to identify governance evaluation tools. Scoping studies provide a picture of the state of research activity in a specific domain (16). One reason for choosing this methodology is that it allows for identifying further evidence that exists about interest and critical concepts and conceptual models, mapping the relevant literature, and identifying the research methods used in the field in question, studying the nature and scope of research studies and the research evidence, identifying research gaps, and determining the feasibility of conducting a systematic review (17).

Scoping reviews require rigorous and transparent methods for the results to be reliable. Therefore, scoping studies follow a structured process and use systematic search but do not suffer from some limitations of systematic reviews (e.g., appraising the quality of reviewed articles). As a result, they cover a broader range of studies in a shorter period, which speeds up the research process. Therefore, scoping review is an appropriate choice when available and urgent evidence is needed on a specific subject (16, 18).

This scoping review was conducted using O’Malley and Arksey’s methodological framework, comprising six stages: identifying the research question; identifying relevant studies; study selection; charting the data; collating, summarizing and reporting the results; involving key stakeholders to inform and validate the study findings (18).

Medline, EMBASE, ProQuest, Scopus, Cochrane Library and Science Direct databases, and Google and Google Scholar search engines were used to find published health system governance evaluation material. The English keywords Health system governance, Health system stewardship, Governance, Stewardship, Health system, Healthcare system, Health system strengthening and Healthcare reform based on Mesh and their Persian equivalents were used. The search strategy for databases is in Table 1, changed according to the search method of each database. Moreover, we manually searched for the obtained documents and extracted the relevant documents.

Table 1:

Database Search Guide

Governance	AND	Health system
OR		OR
Stewardship		Healthcare system
OR		OR
Health system governance		Health system strengthening
OR		OR
Health system stewardship		Health system strengthening
		OR
		Healthcare reform

The criteria for entering the documents in this study were all Persian and English research studies and reviews up to Dec 31, 2021, which pointed to the valuable tools of health system governance. Exclusion criteria for this study included studies published in other languages and after 2022.

The title and abstract of each article were reviewed independently by two reviewers. The full texts of the studies were then retrieved and independently assessed for inclusion or exclusion by the reviewers. Disagreements on the eligibility of studies were resolved either through discussion between the reviewers or by a third party.

The obtained documents were screened and evaluated using the title, abstract and full text. We used a valid checklist for reviewing reviews and research articles to evaluate the studies. The minimum and maximum scores were 1 and 15, and the minimum acceptable score was 10 (19). Finally, we entered the final full text into the information extraction form. The information extraction form included sections such as authors’ information, article title, year of article publication, purpose of the study, and health system governance evaluation criteria. The PRISMA flow diagram was used to select the included studies.

It analyzed the data using the qualitative approach using Braun and Clarke’s framework for thematic analysis: familiarizing oneself with data; generating initial codes; searching for themes; reviewing themes; defining and naming themes; producing the report (20). We observed ethical considerations in all the stages of the research, including a commitment to interpret the information without bias.

Ethical Consideration

This study is part of a PhD dissertation approved by the Ethical Committee of the Tehran University of Medical Sciences - Iran. Ethical code: IR.TUMS.SPH.REC.1398.308.

Results

By 2022, 30 documentaries have designed tools to evaluate health system governance (Fig. 1). Most health system governance studies were conducted in 2012 and 2011 (Fig. 2).

Fig. 1:

The process of reviewing databases and finding studies

Fig. 2:

Frequency distribution of governance evaluation tools by publication year

About 46.67% of documents were articles published in scientific journals, and 53.33% were as official reports of reputable international organizations. Moreover, 96.67% of the documents were published in English and 3.33% in Persian.

Among the proposed tools, 11 specific tools have been designed for governance evaluation, while others have governance as a component of health system evaluation. To develop governance evaluation items, in some tools, the items describe things that are considered “good” actions of governance or use the existing theories to develop primary and secondary measures of health governance, then suggest some “good” actions of governance. In addition, 12 tools assess governance at the health ministry level, three at all levels of the health system, two at the implementation and/or service delivery level, and other studies described a framework for governance evaluation and improvement.

All the identified studies provide components or indicators for governance evaluation, except for (21), which studied governance at three levels without introducing indicators or dimensions. The evaluation tool proposed by the WHO in 2007 assesses governance in the pharmaceutical sector (5), Lewis and Pettersson provide a tool for assessing governance in service delivery (22), Mutale assesses health governance in primary care (23), Kaplan et al. assess human resource governance (24), and Sharma et al. (25) provide a tool for governance evaluation at the district level (23). Other tools are developed at the macro-level or cover all the health system levels. The next step classified governance evaluation measures into indicator and dimension categories (Table 2).

Table 2:

Tools to evaluate the governance of the health system at different levels

Row	Author (yr)	Evaluation criteria
		Dimensions	Indicator
1	Murray (2002)(35)	Overall system design, Performance assessment, Performance assessment, Intersectoral advocacy, Regulation	Has not introduced an Indicator.
2	Islam (2007)(27)	Information/Assessment capacity, Policy formulation and Planning, Regulation	Social participation and system responsiveness, Accountability
3	WHO (2007)(5)	Policy Guidance, Intelligence and Oversight, Collaboration and Coalition Building, Regulation, System design	Accountability
4	WHO (2007)(36)	Registration of Medicines, Control of Medicines Promotion, Inspection of Establishments, Selection of Essential Medicines, Procurement, Distribution	Has not introduced an Indicator.
5	WHO (2008)(30)	Has not introduced a Dimension	Policy index, Outcome Indicators.
6	Tbilisi (2008)(37)	Formulate Health system strategies and policies to ensure achievement of goals, Apply intelligence, Exert influence through coordination with partners and advocate for better health, Ensure adaptability to changing needs (e.g. augmenting system capacity in response to changing needs), Mobilize legal, Regulatory and policy instruments to steer health system performance	Ensure good governance supporting achievement of health system goals
7	Brinkerhoff (2008)(38)	Information, reporting and lobbying, Compact (directives, Oversight and resources), Client power (technical input and oversight), Services	Responsiveness, Voice (preference aggregation)
8	Siddiqi (2009)(15)	Strategic vision, Intelligence and Information	Participation and consensus orientation, Rule of law, Transparency, Responsiveness, Equity and Inclusiveness, Effectiveness and efficiency, Accountability, Ethics
9	Lewis (2009)(22)	Standards, Incentives, Information	Accountability
10	Saved off (2009)(39)	Authority, Information, Motivation, Thinking and policy design	Accountability, Transparency, Control of Corruption, Responsiveness, Equity, Efficiency
11	WHO (2010)(26)	Has not introduced a Dimension	Policy index.
12	Omaswa (2010)(40)	Policy making, Financing and Resource Mobilization, Standard setting/Regulation (monitoring and oversight), Collecting and Disseminating information, Support for research and training, Technical Assistance/Capacity Building, Direct (or contract) Management, International liaison	Has not introduced an Indicator.
13	Baez-Camargo (2011)(28)	Strategic vision and Systems design	Participation, consensus orientation, Accountability, Transparency, Control of Corruption, Responsiveness, Equity, Efficiency
14	Kirigia (2011)(29)	Public health leadership and Management, Effective internal and external partnerships for Health, Macroeconomic and Political stability	Community participation and responsiveness, Horizontal and vertical equity in health systems, Efficiency in resources allocation and use, Accountability and transparency in health development, Ethical practices in health research and service Provision, Rule of health-related laws, Evidence-based decision-making
15	Mikkelsen-Lopez (2011)(7)	strategic vision and policy design	Transparency, Control of Corruption, Accountability, Participation and Consensus Orientation
16	Veillard (2011)(41)	Define vision for health and strategy and policies to achieve better health, exert influence across all sectors and advocate for better health, Make use of legal, regulatory and policy instruments to steer health system performance, Compile, Disseminate and apply appropriate	Ensure good governance supporting achievement of health system goals, Ensure alignment of system design with health system goals
17	Health Systems 20/20 Version 2 (2012) (14)	Client power, Service delivery, Information, Reporting and lobbying, Compact (Directives, Oversight and Resources)	Government responsiveness, Voice (preference aggregation)
18	Olmen (2012)(42)	Coordination, Regulation, Policy guidance	Accountability
19	Mutale (2012)(23)	Regulation &Oversight, Intelligence, Vision	Transparency, Community participation, Accountability
20	Council of Europe (2012)(43)	Organizational arrangements	Accountability, Transparency, Participation, Equity, Quality, Effectiveness, Sustainability, Efficiency, Responsiveness, Integrity
21	Wendt (2012)(44)	Formulating policy/strategy, Aligning and Coordinating action, Regulation the health system, Vision/direction for the health sector, Technical leadership	Facilitating social participation in management processes, Holding health system actors accountable (inform, justify, sanction), Voice, Accountability, Equity/fairness, Alignment/harmonization, Commitment, Strong private sector partnerships
22	Smith (2012)(15)	Setting priorities, Monitoring performance	Holding into account
23	Kaplan (2013)(24)	Information, Strategic vision	Accountability, Transparency, Efficiency, Equity, Responsiveness, Voice and Participant
24	USAID(2013)(45)	Client power, Service delivery, Information, Compact	Voice, Government responsiveness To stakeholders
25	Barbazza (2014) (46)	Partnerships, formulating policy/strategic direction, Generating information /intelligence, Organizational adequacy/system design, Regulation, Coordination, Collaboration, Communication, Control	Participation and consensus, Transparency, Accountability
26	Abimbola (2014) (21)	Has not introduced a Dimension	Has not introduced an Indicator
27	Greer (2016)(47)	Capacity	Transparency, Accountability, Participation, Integrity
28	Mosadeghrad (2016)(33)	Organizational Structure, Inter-sectoral Communication and Cooperation, Regulation, Policy-making and Planning, Stewardship, Monitoring and evaluation, Customer protection	Transparency, Democracy, Rule of Law, Control of Corruption, Ethics, Accountability, Effectiveness, Efficiency, Equity, Sustainability
29	Sharma (2017)(25)	Has not introduced a Dimension	Participation and Responsiveness, Transparency and Fairness, Effectiveness and Efficiency, Accountability
30	Hamra(2020)(3)	Has not introduced a Dimension	Participation, Accountability, Transparency, Use and generation of information, Responsiveness

As well as, most governance evaluation frameworks have used a qualitative method to conduct the evaluation. The data extracted from the studies included in this study are presented in the table below. (Table 2).

In the obtained document, 39 components of governance evaluation were identified and grouped into five dimensions of health policy, strategic planning, organization, and control (Table 3).

Table 3:

Components of health system governance

Main Components	Sub-Components
Health Policy	Problem Definition, Regulation, Conflict of Interest Management, Control of Corruption, Prioritization, Health Policy Formulation and Policy Guidance And
Strategic Planning	Situation Analysis, Determining the Health System Vision, Designing Health Strategies, Formulating A Plan and Setting Goals
Organizing	System Design, Focus and Decentralization, Alignment and Coordination of Actions, Communication, Coalition Building and Capacity Building
Stewardship	Steering, Exert Influence, Health Management, Directing, Advocacy, Internal and External Cooperation, Customer Protection, Attention to People's Needs, Attention to Vulnerable Groups and Community Empowerment
Control	Performance Monitoring, Monitoring, Evaluation, Accreditation, Performance Management, Performance Comparison with Objectives, Implementation of Corrective Actions, Licensing and Complaint Management

The most widely used indicators of health system governance based on existing studies. Thus, indicators of accountability, participation, transparency, justice, efficiency, accountability, corruption control, effectiveness, ethics, the rule of law, and sustainability can be used to evaluate the health system governance (Fig. 3).

Fig. 3:

Frequency of the most frequently evaluated indicators of health system governance

Discussion

We aimed to identify the tools for evaluating governance in the health system. Over 40% of the governance evaluation tools were proposed in 2011 and 2012. With the introduction of steward-ship as the critical function of the health system in the World Health Report 2000(4), researchers began developing tools for assessing health system governance. As this concept became more clarified with the introduction of the term ‘governance’ in the World Health Report 2007(5) and some of its components in the 2010 WHO handbook “Monitoring the building blocks of health systems”(26), there has been an increasing interest in developing governance evaluation tools. As a result, most evaluation tools were developed in 2011–2012.

Two approaches have been used in the evaluation tools. In the first approach, governance is one component of a broader evaluation tool, such as the one proposed by Islam for assessing health system performance, which assesses governance along with other health system building blocks, such as financing and service delivery. Islam uses the World Bank’s evaluation framework to develop indicators and items to assess health system performance and employs document review and stakeholder interviews to collect information (27). In the second approach, a specific tool has been developed to assess the health system governance. One example is the tool developed by which assesses governance at three levels (national, policy, and operational). This tool introduces ten principles for governance, each broken down into several domains, and each domain is assessed using several broad questions (15).

In addition, two approaches have been used to develop governance evaluation items. In the first approach, the items describe actions that are considered “good” governance, and evaluation is based on comparing the existing performance with intermediate goals and outcomes. In this approach, due to the lack of sufficient evidence, it is not possible to make rational judgments about the content or quality of governance activities, such as the WHO evaluation tool. This approach's lack of sufficient evidence does not allow for rational judgments about the substance or quality of governance activities. One example is the evaluation tool developed by the WHO (26). This tool proposes ten outcome-based items for governance evaluation, and any item that exists represents a good outcome.

The second approach uses the existing theories to develop primary and secondary measures of health governance. It is based on a set of good governance indicators proposed, and their relation to intermediate goals and outcomes is assessed. For example, ten principles propose for governance evaluation, each comprising several domains (22 domains), and 63 questions are used to assess these principles at different health system levels. Finally, the weaknesses and strengths of health governance are assessed regarding the health system goals (15).

Some reviewed tools have been designed to assess governance at all the health system levels, considered an advantage; although governance and its mechanisms are developed at the macro (national) level, its implementation depends on individuals at lower levels of the health system. Examples include the tools developed by (15). The advantage of multi-level evaluation tools is that they consider the critical role of actors at different health system levels. In some tools, some requirements must be met before the evaluation begins, such as the tool developed by Baez-Camargo and Jacobs, in which first, a governance problem must be identified in the health system for an evaluation to be performed (28).

Many tools for assessing health system governance use a qualitative approach to evaluate with the assumption that governance results from interactions among different actors in the health system. These tools assess governance by examining the reasons for and levels of interaction among these actors. Using such a qualitative approach can provide helpful information about governance for relevant officials. On the other hand, some frameworks propose a mixed method that, for example, requires calculating indicators, performing in-depth interviews, using data from other studies, reviewing documents, etc. The limitation of using qualitative and mixed methods data does not allow for cross-country comparisons or comparing results in a single country. Among the identified tools, only the one developed by Kirigia and Kirigia uses a quantitative scale for evaluation and proposes indicators for governance (29), used for cross-country comparisons and comparison. However, this tool is tailored to the conditions of African countries as it uses the Ibrahim Index of African Governance, which does not apply to other countries. This tool assigns a total score to governance in each country. This framework consists of 10 functions and 42 sub-functions. A set of questions are developed for measuring each function, rated on a scale from 0% (very poor) to 100% (excellent), and the arithmetic mean of all the items is calculated as the total governance score.

Some of the limitations of the identified governance evaluation tools include value-based governance evaluation criteria, failure to provide recommendations and solutions for achieving more effective governance, the reliance of the majority of evaluation tools on qualitative approaches to data collection, and failure to provide a standard list of indicators for cross-country comparison or comparison of items over time.

Most governance frameworks do not directly contribute to the effectiveness of health system governance in a country; instead, they provide a picture of the current state of governance in the health sector using quantitative and qualitative indicators. This is useful as it can highlight potential weaknesses (such as the lack of a list of essential drugs) and obstacles but does not offer solutions or interventions to increase the effectiveness of governance. Nevertheless, these evaluation tools inform health system stakeholders about problems and guide them in designing effective interventions. In addition, the existing evaluation tools rely more on qualitative than quantitative methods. Only the tool developed by Joses Muthuri Kirigia and Doris Gatwiri Kirigia (29), and the 2008 Governance Evaluation Toolkit by the WHO (30) use quantitative methods. However, the former is tailored for African countries and does not apply to other countries. Besides, the WHO toolkit primarily focuses on specific diseases and does not cover many governance components. Another limitation of the existing tools is that they do not provide a standard list of indicators. Thus, they cannot be used as a quantitative framework to compare countries or changes in the components of governance over time. Frameworks such as the 2010 WHO (26) and Lewis and Pettersson (22) allow for cross-country comparisons at the international level. However, while this information is helpful to donors and international organizations, it is debatable whether it will be helpful to health care providers who may already be aware of governance problems in their health systems or whether it is better to suggest why, where, and how to design and implement appropriate interventions.

Health policy, strategic planning, organizing, stewardship and control were proposed to evaluate the health system governance. Health policy-making is formulating, implementing and evaluating policies to solve community health problems.

Health policy is a set of guidelines developed by policymakers and senior managers of the health system in the areas of financing, production of resources and provision of health services to ensure, maintain and promote the health of the community and guide the decision-making of low-level managers (31).

Strategic planning is a key element of health system governance. Health system leaders and decision-makers must have a broad and long-term vision for the community’s health along with ways to improve the functioning of the health system and, ultimately, people’s health (32). Therefore, the missions and tasks of the health system must be clearly defined. Then, the senior managers of the health system should design plans for implementing policies according to the analysis of the internal and external environment of the health system to achieve the health system goals. Finally, according to the organizational structure of the health system, action plans should be formulated and provided (33).

Proper organizing of the health system, including determining the appropriate organizational structure and division, grouping and proper coordination of components of the health system, has a significant impact on its productivity (34).

The Ministry of Health is the steward and servant of the health system and is responsible for administering, directing and steering health care programs. It should spend financial resources to ensure the public interest and improve their well-being. Effective stewardship increases transparency, participation, justice, efficiency, accountability and responsibility of the health system. Health policies and strategies should cover all activities related to service delivery, resource generation and funding. In this way, the health system as a whole could achieve its goals.

Evaluation and control are other dimensions of governing the health system. A committee in the Ministry of Health should be responsible for monitoring, evaluating and evaluating the health system. The health system goals should be continuously evaluated, and corrective action should be taken. Also, health system policymakers and senior managers should evaluate the impact of implemented health policies and programs on public health. Indicators such as the role of law, control of corruption, accountability, participation, equity, transparency, effectiveness, efficiency, resilience, and ethics could be used to evaluate health governance dimensions.

Good governance leads to the development of appropriate health policies and programs, appropriate coordination and participation inside and outside the sector, guidance and directing of health programs, and monitoring and evaluation the health system's performance to achieve health goals. Governance quality affects the capacity and ability of the health system to provide high-quality, safe and effective health services and leads to the stability of the health system. It is simplistic to expect professional, committed, accountable and responsible managers and leaders of the health system and to have health care providers who provide quality and affordable health services. Good governance leads to a sustainable and resilient health system.

The limitation of this study was the lack of access to specific databases. Moreover, only English and Persian studies were included in this review.

Conclusion

Governance is a complex and multidimensional concept that is evolving. Different tools have been used to evaluate governance in the world, and each of the governance evaluation tools has different components suitable for governance evaluation. However, these tools cannot fully evaluate governance and have shortcomings. Analysis of the tools showed that health policy, strategic planning, organization, and control are the crucial dimensions of health system governance. The indicators of accountability, participation, transparency, justice, efficiency, responsibility, corruption control, effectiveness, ethics, the rule of law, and sustainability could also be used to evaluate the health system’s governance.

It is necessary to strengthen the mentioned dimensions to achieve good health system governance. The health system’s performance will be strengthened by evaluating it and identifying the improvement needs. The health of the community will be improved.

Journalism Ethics considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.