Frameworks of Performance Measurement in Public Health and Primary Care System: A Scoping Review and Meta-Synthesis

Abstract

Background:

This study was conducted to synthesize the evidence on the dimensions of performance appraisal of the public health and primary care system through a scoping review and meta-synthesis.

Methods:

The review conducted systematically in 2018 with a scoping review approach. To identify pertinent studies, the following electronic databases were systematically searched until December 20, 2017: Cochrane, ISI Web of Science, PubMed, Scopus, Science Direct, and Embase. Reviewing the studies found on the search bases was carried out in three stages by two persons individually. According to refined studies, the data were extracted to meet the objectives and respond to the research questions. The thematic analysis was used to identify and categorize the dimensions of performance measurement.

Results:

Using this process, 20 studies were eligible for our research. The critical points in measuring the performance of the public health field were classified into eight main domains including leadership and stewardship, funding, resource generation, service delivery, quality, accessibility, efficiency/productivity, and community health status. The differences in measurement frameworks are inevitable. One reason for the differences in the health system performance measurement framework is the differences in the data or data collection, analysis, and reporting. Performance measurement in the field of health, especially primary care, was a multidimensional issue.

Conclusions:

Each of the main dimensions had several sub-criteria, indicating the broadness and complexity of the performance of first-level care providers. Single-dimensional performance measurement could underpin incorrect policies and decisions.

Introduction

Rapid changes in today's world have affected organizations' activities so that the lack of awareness of these changes will lead to exposure to some complicated crises.[1,2] These rapid and widespread changes have also affected health systems, which are considered as one of the important factors of development and social welfare.[3,4,5] One of the main consequences of these changes is their impact on people's health. Therefore, it is essential for health service providers to enhance their performance to respond to these changes.[3] For this reason, improving the performance of health systems worldwide is considered by governments, investors, and providers.[6] The key to improve performance is to measure performance.[7] Performance measurement is itself some part of the strategic process of performance management.[8]

Focusing on achieving good and sustainable outcomes, the process of performance management considers internal and external factors and achieves certain strategic and measurable goals. It also empowers organization leaders to do monitoring and find out how to achieve their goals.[9] Performance measurement is a great movement around the world that provides some feedback for learning and can help educate managers about the behavior and function of employees.[10] Meanwhile, regarding international health goals, there is interest in increasing the performance measurement of health systems in different countries (especially in developing countries).[11]

Some of the reasons for the increased interest in performance measurement in health systems are the increase in costs, technological advancements, the growth of elderly populations, health market failures, poor quality and deviations of execution, medical errors and injuries, lack of accountability and responsibility, unfairness, and uncertainty.[12] Therefore, governments should use tools for monitoring and evaluating the systems' performance and making more informed decisions about financial resources, organizations, and health systems policies,[11] as international organizations such as the WHO and the OECD have played an international role in encouraging the measurement of health systems' performance.[12]

Although the issue of health systems evaluation has a long history, the first attempts to regularly measure the performance of healthcare systems were started by the WHO 18 years ago.[13] Since then, increased attention has been paid to performance measurement in the organizations that provide health services.[14] So far, many efforts have been made over the past years to create a performance measurement framework with regard to the characteristics of health systems.

Various performance assessment frameworks are designed and used for health systems and their subsets by international organizations and researchers. Some of these models include balanced scorecard, data envelopment Analysis, Donabedian's framework, Starfield's model, health system performance assessment (HSPA) framework, WHO building blocks framework, WHO health systems functions, control knobs framework, systems thinking framework, and health care quality indicators (HCQI) framework of organization for economic co-operation and development (OECD).[15,16,17,18,19] The existence of an appropriate and comprehensive evaluation framework for health system performance can be useful as a decision-making tool for policymakers in the health sector.[20]

The lack of an integrated and universally-accepted framework for measuring the performance of health services has led to different studies on the performance of health organizations from different dimensions.[8,21] Therefore, given the fact that there is no comprehensive agreement on the performance measurement of organizations providing health services, especially primary care, the development and use of a combination of methods, frameworks, and indicators to measure the performance of these organizations can provide a comprehensive and complete perspective of their capabilities.[22,23] In this regard, the synthesis of the existing knowledge can provide comprehensive evidence to help managers with planning and decision-making. Doing a comprehensive review and summarizing all pertinent studies on a research question, knowledge synthesis interprets the results of those studies in a general framework of evidence.[24] Given that summarizing and issuing the results of research is one of the main objectives of field reviews,[25] this study was conducted to synthesize the evidence on the dimensions of performance appraisal of the public health and primary care system through a scoping review and meta-synthesis.

Methods

This is a review conducted systematically in 2018 with a scoping review approach. In this study, Arksey and O'Malley's approach (2005), as well as the complementary recommendations of Levac (2010),[25,26] were used to conduct a scoping review and identify the dimensions of the performance measurement of public health and primary care.

Data sources and search strategy

Too identify pertinent studies, the following electronic databases were systematically searched until December 20, 2017: Cochrane, ISI Web of Science, PubMed, Scopus, Science Direct and Embase. To ensure that the study was not repeated, the Cochrane database was searched initially. The literature search was carried out by reviewing the references of the selected studies. The researcher also signed up on the above-mentioned sites and enabled the alert option to get informed of the new studies related to the topic. The keywords were selected after an initial search and consultation with the librarian. The search strategies are presented in Appendix 1. The search for literature was done by a library specialist. The search results for review and screening were entered into the EndNote v.X7 software by the research team.

Study selection: (Inclusion criteria, screening)

Reviewing the studies found on the search bases was carried out in three stages (title, abstract, and full text) by two persons individually. In the event of disagreement, the final decision was reached through an agreement, but if no agreement was reached, a third-party's comments were used. The inclusion criteria including English language, after 2000, public health and primary health care studies, performance assessment studies, studies to evaluate, the performance of systems or organizations, and use of assessment models in the study. The reason for choosing 2000 as the base year was the first efforts for the performance evaluation of health system started at that year.[13]

In addition, exclusion criteria were studies before 2000, studies of other languages, studies on the field of treatment (hospitals, etc.), staff performance assessment studies, studies on patient evaluation and disease and commentary studies, Letter to the editor, Book, and Report. The studies were screened using the EndNote v.X7 software. Because it was not conventional to do the quality assessment of the studies in scoping reviews,[27] the quality of the selected articles was not assessed in this study.

Data abstraction

According to refined studies, the data were extracted to meet the objectives and respond to the research questions. To do so, a preliminary form was first designed and the data from 2 studies were extracted. The form was then reviewed and used for other articles. According to this form, the authors of the article, years, countries, type of study, research objectives, setting, and indices used were extracted. At this point, one of the authors extracted the required data from the selected studies and the second author examined the data. The form was completed in the Microsoft word software for each of the studies.

Synthesis of data

The thematic analysis was used to identify and categorize the dimensions of performance measurement. Thematic analysis is an inductive method for analyzing qualitative data, which analyzes and understands a particular phenomenon by using open coding to create themes and sub-themes.[28] Extracted data were analyzed using MAXQDA-11. In this study process of thematic analysis included 6 steps: (1) Familiarizing with data: transcribing data, reading and rereading the data, and writing down initial ideas; (2) Producing primary codes: coding extracted data among whole data set and collating related data to every code; (3) Searching for themes: collating codes into possible themes and gathering all related data to every possible theme; (4) Reviewing themes: checking if the themes work in relation to the coded extracted and the whole data set, producing a thematic map; (5) Defining and naming themes: ongoing analysis for refining the specifics of every theme and the whole story that the analysis tells, producing clear definitions and names for every theme; and (6) Generating the report: the final analysis of selected extracts and generating a report of the analysis.[29]

In this study, two of the researchers independently categorized them from conceptual similarities. Focusing on resolving the disputes, they compared their categorizations next. The disputed and uncategorized cases were provided to a third person who was asked to deductively group the uncategorized findings using the defined categories and themes. Then, the third reviewer's comments were compared with the previous categorizations. The disagreements were discussed and the results were ultimately grouped. In the end, the results of the analysis were reviewed and confirmed by two other researchers to assess the understandability and co-ordination of the themes and sub-themes.

Results

In the initial search, 917 articles were found in scientific databases through manual search, and 456 ones were reviewed after excluding repetitive and unrelated articles. In the second phase, 297 studies were reviewed according to their abstracts. Thereby, 198 articles were excluded owing to the lack of the inclusion criteria (99 studies were selected). Finally, reviewing the full texts of the remaining studies indicated that 19 studies were eligible for our research [Figure 1].

Figure 1.

PRISMA flow diagram for selecting article

Of the 19 studies selected, 5 were multi-country studies. The study population of this studies included OECD countries, low and middle income ountries (LMICs), European Union (EU) countries, 173 countries, and Netherlands and Ontario, Canada. The scope of most of the included studies was the national health system (63%). Scopes of other studies included primary care at national level (2 studies), service provider organization level (2 studies), and primary care in rural areas (2 studies). One study also examined national and international levels simultaneously. A summary of the features of those studies is presented in Appendix 2.

On the basis of the analysis carried out, the critical points in measuring the performance of the public health field were classified into 8 main domains including leadership and stewardship, funding, resource generation, service delivery, quality, accessibility, efficiency/utilization, and community health status [Table 1].

Table 1.

Taxonomy of critical points to performance measurement in public health and primary care system

Domains	Sub-domains/categories		References
Leadership and Stewardship[6,30,31,32]	Strategic planning[33]	Vision	[34]
		Mission	[14]
		Forming health policies, determining prospects and strategies	[6]
	Approaches for controlling and adjusting
	Accountability system and inter-sectoral co-operation
	Organizing healthcare service delivery
Funding[6,31,32,34,39]	Fair financing		[11,31]
	Costs/Expenditure		[10,13,17,30]
	Revenue collection		[6]
	Integrating and allocation the resources to state budget
	Protection from financial risks[6,11]	Demographic characteristics
		Economic features
		Consumptions and healthcare costs
Resource generation[6,11,31]	Health information system		[6,30,32]
Human resources			[6,30,32,34]
	Medicine and Medical supplies		[6,32]
	Growth and Learning	Collecting and using knowledge	[6]
		Use of data audit and feedback processes	[33]
		Innovation adoption
		Training and continuing education for workforce
Service delivery[6,31,32,34]	Adult Health Services (AHS)		[32]
	Child Health Services (CHS)
	Antenatal Services[45]
	Outcomes		[14]
	Capacity for service provision		[34]
	Fairness in healthcare services delivery		[6]
	Continuity		[50]
	Coordination		[13,30,50]
	Comprehensiveness	Service available	[30,50]
		Service provided
Quality[10,11,13,17,31,33,51]	Quality of care (Clinical quality)		[11,33,61]
	Management quality		[33]
	Wastage/errors		[51]
	Effectiveness of health services		[10,11,13,17,30,31,48,49]
	Safety		[10,13,17,30,31]
	Patient and community[34]	Patient/family satisfaction	[11,13]
		Family-centeredness	[50]
		Community orientation	[50]
		Community support	[33]
	Responsiveness/patient centeredness[10,11,31,47]	Autonomy	[47]
		Attention
		Communication
		Amenities
		Choice
		Confidentiality
		Respect
	Responsiveness to people’s non-medical expectation		[31]
	Appropriateness[30]	Adults receiving recommended screening and preventive care	[51]
		Children receiving recommended screening and preventive care
		Others
Access[10,11,13,17,31,33,48,51,61]	Equitable access to healthcare services	Financial factors	[6]
		Geographical factors
		Individual factors
		Structural factors
	Financial access		[33]
		Information access
		Linguistic access
		Physical access (Structural)
		Service availability/allocation
	Rural public health system coverage		[49]
		Equality of rural public health services
	Cultural competence		[50]
	Acceptability		[51]
Efficiency/Utilization[11,13,30,33,48,51,61]	Administrative efficiency		[11]
	Costs and productivity
	Adequacy of funding
	Direct inputs		[52]
	Direct output
	Contextual factors
	Patient or procedure volume per time period		[33]
	Patient or procedure volume-general
	Patient or procedure volume relative to capacity
	Patient or procedure volume relative to population
	Patient or procedure volume relative to population health characteristics
	Patient or procedure volume relative to the need of the patient
	Service usage relative to income group
	Cost to service ratios
	Staff to service ratios
	Additional costs		[51]
Health status and community[11,30,31,61]	Improving the health of the population		[31,48]
	Healthy lives		[51]
	Demographic and socio-economic factors		[31]
	Determinants of health		[31]

Leadership and stewardship

A total of 7 studies focused on the issue of leadership and stewardship and its subsets. The results showed that in the leadership and stewardship domain, the studies had examined strategic planning (vision, mission, policies, and strategy), control and adjustment approaches, accountability, inter-sectoral coordination, and organizing of health services.

Funding

A total of 10 studies focused on the issue of funding and its subsets. The results suggested that fair financing, costs and expenses, revenue collection, accumulation and allocation of financial resources, and protecting financial risks were the components of this dimension.

Resource generation

A total of 7 studies focused on the issue of resource generation and its subsets. The results of this study also indicated that human resources, information, medicine and medical equipment, and growth and learning of the employees had been evaluated as health system resources in previous studies.

Service delivery

A total of 9 studies focused on the issue of service delivery and its subsets. This theme contains adult health services, child health services, antenatal cervices, continuity, coordination, comprehensiveness, and so on.

Quality

A total of 14 studies focused on the issue of quality and its subsets. It was found out that several factors had to be considered for quality assessment. In this regard, the researchers addressed a variety of topics for quality assessment, including the quality of care, management quality, errors and losses, safety, accountability, suitability, and service effectiveness.

Accessibility

A total of 11 studies focused on the issue of accessibility and its subsets. The results also showed that accessibility was a multidimensional issue, and the studies had examined it from different perspectives: financial, information, linguistic and physical (Structural) access, service availability/allocation, rural public health system coverage, equality of rural public health services, cultural competence, and acceptability.

Efficiency/productivity

A total of 8 studies focused on the issue of efficiency/productivity and its subsets. It was indicated that efficiency/productivity was one of the aspects of health systems performance measurement.

Community health status

A total of 6 studies focused on the issue of community health status and its subsets. The results of the study showed that the researchers examined community health improvement, healthy living, demographic and socioeconomic factors, health determinants, patient and family satisfaction, and family-centered and community-based services for assessing health status.

Discussion

It was indicated in this study that there were different structures and approaches to assess the performance of primary care. Some of these differences could be owing to the national policies and programs or owing to the priorities and technical differences in the health system of the countries. However, various frameworks experienced by different countries could be valuable to policymakers, health managers, and researchers from other countries. This review attempted to present the dimensions used to evaluate the performance of primary care as a comprehensive set. The results showed that performance measurement in the field of public health, especially primary care, was a multidimensional issue and all its dimensions needed to be considered. Thus, the topics used were classified into 8 main themes, the importance, role, and position of which are discussed in the following.

Leadership and stewardship

The results of this study showed that leadership and stewardship had been introduced into a few performance evaluation frameworks. According to the results, only seven studies had focused on the field of leadership and stewardship in primary care performance measurement.[6,14,30,31,32,33,34] However, the WHO has introduced stewardship as one of the key functions of health systems.[35] Health stewardship that had a close relationship with the concepts of leadership and governance, generally refers to the laws and institutions that shape policies, programs, and activities related to health goals.[36] Makuta and O'Hare indicated a positive relationship between the indicators of stewardship and leadership and the health outcomes and performance rate.[37] However, Lagomarsino et al. believed that the existence of effective leadership in the health system was essential for achieving broad health goals.[38] Besides, Pallas et al. argued that management and leadership strategies for improving organizational performance were more important than the strategies such as creating new skills and knowledge or making a new box in the organizational chart because a new paradigm was needed to determine how the groups within an organization should interact with each other to achieve the organizational goals. One of the interventions of this strategy was said to be the creation of monitoring systems to manage the performance of the organization's executives in response to the organizational goals.[33] Hence, to have a comprehensive framework for assessing the performance of health care providers, it is recommended to take leadership and stewardship into consideration.

Funding

The results suggested that fair financing, costs and expenses, revenue collection, accumulation and allocation of financial resources, and protecting financial risks were the components of this dimension. Our study also showed that 50% of the studies had focused on the financial dimension of primary care when measuring primary care performance. In their study, Shahri et al. pointed out that financial resource management played an essential role in measuring the performance of health systems. In their opinion, the three functions of financial management were resource collection, resource accumulation, and purchase of services.[6] The WHO also considered these three functions essential to ensure people's access to health services.[35] In a study by Dos Santos et al., the financial dimension was considered as the most influential dimension of public health systems' performance.[39] Because of the financial system and the costs of providing services build the basis for the efficiency of health systems,[11] the financial system must be regularly and effectively evaluated using validated indicators.

Resource generation

For better performance and continuity of health care provision, the primary care system requires a variety of resources, including human, physical (equipment and facilities), and informational resources.[2] Edward et al., who used the balanced scorecard framework for their assessment of health systems and considered the resources such as medications, functional equipment, and training as the components of capacity dimension for providing services. They also evaluated staff as a separate dimension.[34] Some studies, such as the one by Mutale et al. in which the WHO framework was used, evaluated three components of resource production (human resources, health information, and medical equipment) separately.[32]

Human resources can be considered as one of the most important ones; because by utilizing health technologies, it plays a key role in managing and providing services and promoting community health.[40,41] However, 60% to 80% of the total health care costs are usually manpower costs. Most importantly, quality, efficiency, effectiveness, availability, and acceptability of health services depend on the performance of those who provide these services.[42] Therefore, given the fact that achieving the goals of health systems, especially primary care, depends on the efficient and capable manpower; assessment and improvement of their performance can lead to improved service quality and increased satisfaction of the clients. Mutale et al. pointed out that interventions in the field of human resources would have an incremental effect on leadership and stewardship, provision of services, financial systems, and health information.[32]

Information resources are another important sources in the health sector, so that having a comprehensive health information system can have many potential advantages in terms of financial benefits and improvement of healthcare quality.[43] Furthermore, the existence of a health information management system helps to determine the appropriate indicators to make decisions and improve the performance of the organization.[6] It should also be noted that having a proper information system in the organization will be useful in the process of evaluating and providing the necessary data to measure the performance of the organization.

Service delivery

Assessment of health services delivery was a key dimension in the performance measurement of health organizations. For instance, in the study by Shahri et al., health service delivery with an impact factor of 0.79 was introduced as the most effective determinants of health system performance assessment.[6] According to the conducted analysis, attention was paid to fairness, continuity, integrity, co-ordination, and services for special groups (Like children, mothers, and adults) when assessing the provision of primary care.

Providing health services to the community is the main process in any health care system that is needed to achieve the main goal of the system.[35] However, providing services represents an immediate (primary) output from the health system's inputs, and a fundamental input to the health status of the community.[44] For this reason, providing health services is the backbone of the health system.[45] Therefore, it is also considered as one of the key pillars of performance evaluation frameworks. For example, one of the 6 main dimensions of the performance system designed to manage the provision of primary care services in Afghanistan was the provision of services.[34] Hence, considering the importance of strengthening the delivery of health services to achieve the millennium development goals (MDGs),[44] monitoring, and evaluating it is essential and recommended.

Quality

Quality is a very important issue in health systems because poor service quality leads to diseases, disabilities, higher costs, and lower confidence in the health system.[46] This is why the quality of health services is one of the main dimensions of health system performance. The results showed that 60% of the selected studies focused on quality and its subsets. In their study, Pallas et al. reviewed 181 experimental studies of health service providers in countries with middle-toward-down income and suggested that quality had the highest frequency (83%) among performance dimensions.[33] Studies have shown that researchers' viewpoints toward quality are different. For example, in their study, Tawfik Shukor et al. examined the framework of performance measurement system in the Netherlands and Ontario, and called the three dimensions of safety, effectiveness, and accountability as “quality diamonds”. However, Kruk et al. considered quality of care as one of the subsets of effectiveness and classified its indicators into three categories of impact, safety, and continuity.[10,11]

The results of the present study showed that the previous studies had addressed effectiveness, safety, suitability, accountability, errors, clinical quality, and management quality for quality assessment. Accountability, which is one of the primary goals of any health system,[35] has some dimensions that are important indicators of health system performance. In their study, Jacob et al. found a correlation between quality and accountability.[47] There are different perspectives on the role of effectiveness in evaluation frameworks, so that in some studies, it was considered as one of the dimensions of service quality.[10,13,31] However, in the study by Jahanmehr et al., it was evaluated as an individual dimension.[48] Effectiveness has also been used in the frameworks developed by the WHO and the OECD.[12,14]

Quality indicators were first developed to improve patients' outcomes, but now their information is used for monitoring, management, and policymaking.[17] However, Van Den Berg et al. in the Netherlands referred in their study to the rarity of good information on quality according to which patients and insurers could make their choices.[13] Therefore, quality assessment and monitoring, which can help managers, insurers, and patients to make decisions and improve health systems should be properly considered.

Access

Shahri et al. addressed fair access to health services and included financial, geographical, individual, and structural factors.[6] To examine access to health services, Pallas et al. addressed financial, informational, linguistic (cultural), and physical (structural) access as well as allocation/availability of services.[33] Tian et al. evaluated rural public health services and considered two issues of coverage and equality.[49] Pasarin et al. and Gauld et al. separately evaluated accessibility in terms of cultural competence and acceptability of services.[50,51]

From patients' perspective, timely access to a complete set of required services is one of the characteristics of an effective health system.[11] In this regard, primary care which is the base of any health systems should be available to all people in the community.[51] However, in developing countries, access to basic services is still one of the major barriers to improve people's well-being. Hence, this issue needs to be considered by policymakers and analysts.[35] One way to measure people's access to health services is to incorporate it into the frameworks of health system performance measurement. Accordingly, the results of the present study showed that this issue has been well considered in the performance measurement frameworks, so that 11 studies (55%) reviewed had evaluated accessibility. In general, accessibility includes three components: availability, utility, and timeliness.[11] These studies have addressed various aspects of access including financial, physical, cultural, geographical, structural, and informational access.[6,33,48]

Timely access to health services is essential for maintaining lives and minimizing suffering and disabilities.[11] Terner et al. also considered the waiting time for receiving immediate care as one of the indicators of access.[30] Thus, it should be evaluated and measured using valid and accurate indicators, and its barriers need to be eliminated according to the measurement results. It should be noted, however, that access measurement requires attention to manpower and service capacity and whether the services are available.[51]

Efficiency/utilization

To evaluate efficiency/utilization, Pallas et al. used the ratios of costs to services, staff to services, use of services to income groups, volume of patients or procedures to population health characteristics, and so on.[33] Sun et al. categorized performance indicators in three groups: direct inputs, direct outputs, and underlying factors.[52] Kruk and Freedman addressed efficiency in relation to management, costs and productivity, and financing adequacy.[11]

In health systems, efficiency measurement is usually the first step in evaluating the function of the units such as health centers, hospitals, etc.[53] The reason why efficiency measurement in health systems is important is that patient health outcomes can be measured.[54] However, the inefficient use of resources in primary care is related to the number of avoidable hospitalizations.[55] Nevertheless, the results of the study showed that only 40% of the studies reviewed had addressed efficiency in measuring the performance of public health providers. However, efficiency is one of the major criteria in resource allocation processes.[53] Therefore, monitoring and evaluating the efficiency of public health systems at different levels can help to better allocate the resources.

Marschall and Flessa stated that the achievement of low-income countries in the health-related MDGs depended on focusing on efficiency.[56] Efficiency analysis plays an important role in empowering policymakers, managers, and health providers to improve the structures, processes, and outcomes of primary care. However, owing to the unclear boundaries of health service provision, it is a challenging task as well.[57] Efficiency is not only one of the dimensions of the performance measurement framework but also one of the main goals of the implementation of the monitoring and evaluation system.[48] Therefore, although performance measurement in public health systems is complex and challenging, its multiple benefits and implications for health systems have made it necessary and inevitable.

What was said above indicates the importance of efficiency in public health systems. Therefore, special attention should be paid to it by health managers and policymakers. However, some studies might have specifically addressed efficiency, but incorporating it into performance measurement frameworks may lead to better policymaking and decision-making. Hence, to have a comprehensive and effective measurement system, the issue of efficiency should be considered and its indicators need to be monitored.

Community health status

The results of the study showed that the researchers examined community health improvement, healthy living, demographic and socioeconomic factors, health determinants, patient and family satisfaction, and family-centered and community-based services for assessing health status. Because supplying, maintaining, and promoting community health are the main goals of health systems, the extent to which these goals are achieved should be evaluated.[35,58] From the perspective of society, the essential duty of an effective health system is to improve the health status in the country.[11] Besides, the extent to which the services are based on the community or families is considered for the purpose of evaluating community health.[50] Access to community-based primary care is a priority in many countries.[59] Population-based primary care is intended to provide first-contact healthcare services to ensure the continuity of care, ease of movement across the system, and improved system integration.[60] However, measuring the consequences is difficult, and it is not easy to distinguish the degree of health system participation from other factors.[35] However, as improving the health status is considered a measure for health system effectiveness,[11] it is an important part of performance measurement and should be considered.[61]

Limitations of study

This meta-synthesis by search criteria has been deliberately limited to measuring the performance of public health and primary care. Minor measurement criteria such as quality, efficiency, etc., were not included in the search strategy of the present study. Moreover, the findings of this meta-synthesis are limited by searched databases, time, and language of publication. There may be some studies in old literature or other languages that were not entered into this analysis.

Conclusions

According to the results of this study, performance measurement in the field of health, especially primary care, was a multidimensional issue, indicating the widespread and complexity of the performance of first-level care providers. Single-dimensional performance measurement could underpin incorrect policies and decisions. If the dimensions and indicators of measurement are well-defined and tied to management and policy processes, it can help improve performance. However, the differences in measurement frameworks are inevitable. One reason for the differences in the health system performance measurement framework is the differences in the data or data collection, analysis, and reporting. According to the evidence, the dimensions used depended on the measurement model used, the purpose of measurement, the view of the executives, and the participants in the studies. Finally, the results of this study can provide a comprehensive viewpoint about performance evaluation to managers and help them to design a performance assessment framework.

Financial support and sponsorship

This study was funded by the Students Research Committee of Kerman University of Medical Sciences (Grants No. 97000405).

Conflicts of interest

There are no conflicts of interest.

Appendix 1: Search strategy

Databases	Search strategies
Cochrane library	#1(“Performance evaluation” : ti OR “Performance assessment” : ti OR “performance measurement” : ti OR “assessment framework”: ti OR “health system performance” : ti OR “monitoring and evaluation” : ti OR “Primary care Assessment”:ti #2(“health system” :ti OR “Healthcare system”:ti) #3(#1 AND #2)
PubMed	#1(“Performance evaluation”[Title] OR “Performance assessment”[Title] OR “performance measurement”[Title] OR “assessment framework”[Title] OR “health system performance”[Title] OR “monitoring and evaluation”[Title] OR “Primary care assessment”[Title]) #2(“health system”[Title] OR “Healthcare system”[Title]) #3(#1 AND #2)
Scopus	TITLE(“Performance evaluation” OR “Performance assessment” OR “performance measurement” OR “assessment framework” OR “health system performance” OR “monitoring and evaluation” OR “Primary care assessment”) AND TITLE(“health system” OR “Healthcare system”)
Science direct	Title:(“Performance evaluation” OR “Performance assessment” OR “performance measurement” OR “assessment framework” OR “health system performance” OR “monitoring and evaluation” OR “Primary care Assessment”) AND(“health system” OR “Healthcare system”)
Embase	#1 “Performance evaluation”: ti OR “Performance assessment”: ti OR “performance measurement”/exp OR “assessment framework”: ti OR “health system performance”: ti OR “monitoring and evaluation”: ti OR “Primary care Assessment”: ti #2 “health system”: ti OR “health care system”/exp #3 #1 AND #2
ISI Web of science	TI=(“Performance evaluation” OR “Performance assessment” OR “performance measurement” OR “assessment framework” OR “health system performance” OR “monitoring and evaluation” OR “Primary care assessment”) AND TI =(“health system” OR “Healthcare system”)

Appendix 2: A summary of the features of selected studies

ID	Source (Year)	Design/Methods	Country	Scope/Setting	Aim of study
1	Dos Santos, et al. (2015)	Case study	Brazil	Public health system in a Southeastern town	Presents the application of the Analytic Network Process and Balanced Scorecard in the performance evaluation.
2	Mutale, et al. (2016)	Interventional/Mix Methods	Zambia	Three rural district	Present a proposed framework for evaluating a new health system strengthening intervention.
3	Kruk and Freeman (2008)	Literature review	-	Health system	Propose a framework for the assessment of health system performance and review the literature on indicators.
4	Terner, et al. (2013)	Review, survey	Canada	Primary healthcare	Update primary healthcare indicators for use across Canada.
5	Tian, et al. (2013)	Cross-sectional	China	Rural public health system	Providing the up-to-date evidence and a performance assessment toolbox for assessing the trends of public health services accessibility and financial protection in rural china from 2008 to 2010, as well as the current situation about the china’s rural public health system performance.
6	Tawfik-Shukor, et al. (2007)	Cross-sectional	Netherlands and Ontario, Canada	Health system	Comparing health system performance approaches in the Netherlands and Ontario, Canada
7	Jahanmehr, et al. (2015)	System thinking, literature review and interview	Iran	Deputy of health	Design a conceptual framework, according to the policies and priorities of the ministry of health to evaluate provincial health and primary care performance.
8	Handler, et al. (2001)	Expert panel	USA	Public health system	Describes a unifying conceptual framework for the public health system as a way to facilitate the measurement of public health system performance
9	Edward, et al. (2011)	Cross-sectional	Afghanistan	health system	Illustrate the performance trends in delivering the basic package of health services during 5 year period (2004-2008)
10	Shahri, et al. (2016)	Descriptive, analytical, and practical (Questionnaire)	Iran	health system	Determine factors evaluating the performance of health systems based of structural equation modeling.
11	Van den berg, et al. (2014)	Review	Netherlands	Health system	Discuss the development process of the Dutch Health Care Performance Report (DHCPR) including the conceptual approach of assessment the health care system.
12	Pasarin, et al. (2013)	Observation, expert panel, cognitive interview, and questionnaire	Span	Primary care	To obtain versions of the primary care assessment tools (PCAT) - facility version to evaluate primary care in Spanish context and analyze its feasibility, reliability, and validity.
13	Sun, et al. (2017)	Economic analysis of efficiency	173 countries	National health system	Examine the efficiency of national health systems using longitudinal country-level data.
14	Rohova, et al. (2017)	Review article	Bulgaria	International and national level	Review the current state of research on health system performance assessment at international and national level.
15	Pallas, et al. (2012)	Review article	Low and middle income countries (LMICs)	Health service delivery organization	Develop a taxonomy of strategy areas and a conceptual framework for selecting strategies to improve performance of health service delivery organizations.
16	Hofmarcher, et al. (2016)	Review and expert panel	EU member states	Health system	Develop a blueprint of a platform to feature a set of headline indicators and relevant meta-information for HSPA across EU member states.
17	Yakob and Ncama (2017)	Cross-sectional	Ethiopia (Wolaita Zone)	Health system responsiveness at facility level	Innovatively measure health system responsiveness and correlates in the context of HIV/AIDS treatment and care services.
18	Gauld, et al. (2011)	Benchmark	New Zealand	Health system	Develop a national scorecard for assessing health system performance derived from routine data.
19	Carinci, et al. (2015)	Modified Delphi, consensus meeting	OECD countries	Health system	Review and update the conceptual framework, indicator content of the OECD health care quality indicators (HCQI) project.