Identification of common indicators of hospital performance evaluation models: A scoping review

Abstract

BACKGROUND:

Hospitals in developed countries allocate itself about 40% of health-care costs and in developing countries up to 80%. In this study, researchers reviewed the related article in the field of the hospitals' performance evaluation due to the identification of common indicators used in different models with the aim of providing simple list for evaluating hospitals' performance to the researchers who are interested to these issues.

METHODS:

The databases for the search included Medline, Google Scholar, and Scopus. A search strategy leads to the extraction of 403 related articles that after the removal of inappropriate and duplicate articles, 42 studies were selected for the scoping review.

RESULTS:

This scoping review showed that 9 of 42 selected articles use data envelopment analysis model, 6 of them Pabon Lasso model, 3 of them balanced scorecard, 3 of them organizational excellence model, and the remaining articles used key performance indicators to evaluate hospital performance. A process approach was used to categorize the common indicators because the literature review indicated that common indicators mainly evaluate data, process, output, or impact of performance. The findings included 36 input, 39 process, 27 output, and 8 impact common indicators for evaluating performance.

DISCUSSION:

This scoping review of related literature indicated that despite differentiation in the time and place of past studies in the field of performance evaluation models, there are some indicators that are common in most popular performance evaluation models. These simple lists can apply for evaluation of hospitals' performance instead of complicated models.

Introduction

In developed countries, hospitals allocate about 40% and in developing countries up to 80% of health-care costs. Therefore, the results of hospitals' performance in the health-care system are a great priority.[1]

On the other hand, based on the literature review, more than half of national health resources are wasted in different countries, especially in underdeveloped countries, which means limited resources are inefficiently consumed or national income is spent on services that are not proportionate and effective. Therefore, ensuring the quality of health-care services requires evaluating the performance of health-care organizations to alleviate some of the government's financial problems by minimizing waste.[2] Therefore, many health-care providers have stepped to path of performance evaluation in order to achieve their goals and missions.[3]

In fact, these organizations consider performance evaluation models as a way to move toward continuous improvement and valid tool for measuring of the organization's current performance.[4]

On the other hand, an appropriate and applicable evaluation model for hospital performance potentially would lead to more accountability, service quality, and satisfied customer. When a hospital's performance is not evaluated, any claims about its service quality will be unproven, and there is no evidence to improve performance. Therefore, the evaluation and improvement of performance are two sides of the same coin that will lead to continued improvement, and the lack of an efficient performance evaluation will lead to serious problems for hospitals from the aspect of responsibility and performance improvement. Until two decades ago, designing a performance evaluation system was just a mere innovation, but today, it has become an indispensable necessity for any organization. With all these interpretations, the evaluation of performance is very complex and difficult to done, which requires pay attention to all various dimensions of performance that would affect the job results.[5]

The WHO's definition of performance is achieving the desired goals. The excellent hospital performance should be based on professional competence in benefit from the latest knowledge, advanced technology and of course regarding available resources, efficiency of resource, minimum risk for patient, and patient satisfaction. In other words, according to literature review, the performance evaluation is not worthwhile alone and will be valuable if the results of that used to continually improvement of performance.[6]

Today, several models to evaluate the performance of an organization have been introduced such as models of organizational excellence (European Foundation for Quality Management) that evaluates the performance of organizations for establishing total quality management[7] or the balanced scorecard (BSC) approach that transforms the organization's strategic goals into measurable indicators and establishes a balanced distribution across the organization's critical areas. This approach, as a measurement system while operationalizing the organization's vision and strategies, can provide a comprehensive picture of the organization's performance. In other words, a balanced scorecard can play an effective role in realizing the vision and strategy of an organization using a set of financial and nonfinancial indicators.[8]

Other ways to evaluate hospitals' performance are the use of key performance indicators (KPIs). In some studies, the performance of hospitals was evaluated by identifying and measuring KPIs.[9]

Organizations that apply any evaluation's models to measure its performance at first should identify some of the effective indicators. Improving hospital indicators means efficiency and effectiveness in hospital activities, in other words, best use of resources.[10] So far, many studies have been conducted to evaluate the performance of hospitals in different countries. In this study, we have attempted a scoping review of studies carried out over the past 10 years (2018–2008) on the performance evaluation of hospitals using different methods in Iran and other countries to reach common evaluation indicators.

Methods

In recent years, many studies have been conducted to evaluate the performance of health-care organizations in different countries. Therefore, because of the large number of studies in this field, the search strategy limited to the 10-year period (2008–2018). Furthermore, with regard to the purpose of the study, which was a scoping review of studies conducted in the field of hospital performance, the researchers excluded studies that done in other health-care organizations. Therefore, the keywords used to search the related articles were as follows: Performance Evaluation, Hospital, Performance Evaluation Models, and Performance Evaluation Methods [Table 1].

Table 1.

Search strategy

SPIDER tool[1]	Search term
S	Hospital* OR clinic* OR hospice* OR (health AND organization OR center)
PI	Assessment*OR evaluation* OR appraisal* AND (performance*OR function AND operation) AND efficiency*
D/E/R	“Qualitative” OR “quantitative” OR “mixed method*” OR “case study” OR “cohort study” OR “quality assurance”

*=When some words are incomplete

The databases used for the search included Medline, Google Scholar, Scopus, EBSCO Discovery Service, and Web of Science [Table 2]. A search of these databases leads to the extraction of 403 articles that, after the removal of duplicate and non-English articles, 123 articles with screening criteria (keywords in the title or abstract of the articles) were selected. Furthermore, in order to focus on the main purpose of the study, literature that evaluated the results of performance evaluation without using a specific model were excluded. After rescreening, studies that were repeated in the EndNote X7 for windows because of the use of the snowball method in references also were excluded. Finally, 42 studies were extracted for the scoping review [Figure 1].

Table 2.

Inclusion and exclusion criteria

Criterion	Inclusion	Exclusion
Time period	January 2007 and February 2017	Any study outside these dates
language	English and Persian	Non-English
Kind of organization	Hospitals and health centers and health-care organization	No reference to l health care organization
Tool	DEA, BSC, EFQM, Pabon Lasso, lean, JCAH, accreditation standard	Any study without this method
Population and sample	All kind of health-care organization (clinical and nonclinical)	Nonhealth-care organization

DEA=Data envelopment analysis, BSC=Balanced scorecard, EFQM=European Foundation for Quality Management, JCAH=Joint Commission on Accreditation of Healthcare

Figure 1.

Screening process of articles

After the final review of extracted articles, the selected articles were reviewed using the matrix method – a system for organizing, critically evaluating, and synthesizing research articles for scientific review of articles. Inclusion criteria were also used for reviewing and evaluating relevance articles. A scoping review is done to attain to deep perspective of a selected topic, not to evaluate or criticize the best study that have been counducted. Therefore, the studies were reviewed by a researcher, away from any critical review and weighting of the studies.

Results

After screening the studies, 42 studies that met the inclusion criteria were carefully reviewed. The characteristics of these studies are summarized in Table 3 by target population, study location, research method, and time of study.

Table 3.

Descriptive of reviewed articles

Descriptive variable	n (%)
Place
Hospital	39 (93)
Other health-care organizations	3 (7)
Country
USA	10 (24)
Asia	26 (62)
Africa	1 (2)
Europe	5 (12)
Method
Qualitative	4 (10)
Quantitative	38 (90)
Time
Before 2010	6 (14)
After 2010	36 (86)

Out of 42 selected articles, 18 were published in other countries [Table 4] and 24 in Iran [Table 5], 9 of them use data envelopment analysis,[2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]6 of them Pabon Lasso model,[8,18,19,20,21,22]3 of them BSC,[23,24,25]3 of them organizational excellence model,[4,7,26]3 of them used accreditation standards,[27,28,29] and other articles used KPIs to evaluate hospital performance.[1,30,31,32,33,34,35,36,37,38,39,40,41,42] The Pabon Lasso model was used for the evaluation of hospital performance only in Iran in determined time period. In two studies, combined models were used for performance evaluation, and in some studies, performance evaluation indicators have been used; these indicators were hospital mortality rates and readmission rates as performance evaluation indicators.

Table 4.

Descriptive of English reviewed articles

Authors	Country	Years	Performance assessment method
Dong et al.	China	2017	DEA: Systematic review
Karsak et al.	Istanbul	2017	DEA
Glover et al.	USA	2015	Social media
Lacko et al.	Slovakia	2014	DEA
Ash et al.	USA	2012	Statistical issues
Baradach et al.	USA	2012	Commercial website
Stephan et al.	USA	2012	Readmission rate
Renzi et al.	Italia	2012	Quality indicators
Grigorodis et al.	Greece	2011	BSC
Stephen et al.	USA	2011	JCAH
Chung et al.	Taiwan	2010	DEA
Yawe et al.	Uganda	2010	DEA
Abujudeh et al.	USA	2010	KPI
Weng et al.	USA	2009	DEA
Kneenan et al.	USA	2008	Readmission rate
Derrigo et al.	Italia	2008	Empirically derived model and euroscore system
Werner et al.	USA	2007	Mortality rate
Jha et al.	USA	2007	Mortality rate

DEA=Data envelopment analysis, BSC=Balanced scorecard, JCAH=Joint Commission on Accreditation of Healthcare, KPI=Key performance indicator

Table 5.

Descriptive of Persian reviewed articles

Authors	Years	Performance assessment method
Saeedi et al.	2018	Lean
Barati et al.	2017	Pabon Lasso
Jahangiri et al.	2017	MADM
Rajouee et al.	2017	AHP + BSC
Alinezhad et al.	2017	BSC + VFB-DEA
Omidvari et al.	2016	FANP + BSC
Dargahi et al.	2016	Pabon Lasso
Bastani et al.	2016	Pabon Lasso
Khanzadeh et al.	2015	EFQM
Arzemani et al.	2014	Accreditation standards
Tabatabaee et al.	2013	EFQM
Raeesi et al.	2013	Standard of ministry
Rahbar et al.	2013	Pabon Lasso
Azar et al.	2013	LINMAP + SAW + HBSC
Azar et al.	2013	DEA
Parham et al.	2013	EFQM
Bahadori et al.	2012	Systematic review
Khani et al.	2012	DEA
Samadi et al.	2012	BSC
Janati et al.	2012	Expert consensus
Bahadori et al.	2011	Pabon Lasso model
Sheikhzadeh et al.	2010	Delphi
Asadi et al.	2010	DEA + BSC + SERVQUAL
Sajadi	2009	Pabon Lasso

BSC=Balanced scorecard, DEA=Data envelopment analysis, EFQM=European Foundation for Quality Management, SAW=Simple additive weighting, HBSC=Hierarchical Balanced Scorecard AHP=Analytical Hierarchy process, MADM=Multi Attribute Decision Making, VFB=Virtual Frontier Benevolent, FANP=Fuzzy Analytic Network Process, LINMAP=Linear Programming for Multidimensional Analysis of Preferences, SERVQUAL=Scale for measuring service quality

After reviewing the indicators used in hospitals' performance evaluation-related literature review, a process approach was used to categorize the common indicators in most applied performance evaluation models because the literature review indicated that common indicators are main kind of indicators to evaluate data, process, output, and impact of performance. This category is summarized in Table 5.

Discussion

Much has been echoed in the literature about the importance of understanding performance. According to industrial models of production, where the efficiency of production is paramount, some researchers have proposed that health care could be viewed under the same magnifying glass as the production of other goods.

Nevertheless, in many countries, the hospital environment has completely changed in the past 20 years compared to industrial zone, so health-care organizations have been forced to apply continued performance improvement approaches to survive in competitive environment. Therefore, seeking to comprehensive, reliable, strategic, and flexible model to evaluate performance has become an important priority and an undeniable responsibility for them. As mentioned above, health-care management and health-care industry have been one of the popular and complex topics that many researchers and professionals have focused on. In this study, researchers reviewed studies conducted over 10 years (2007–2017) with the aim of extracted common indicators of evaluating hospital performance.

For as much as in hospitals such as other organizations, services are delivered through determined processes and the researchers decided to summarize and categorize the common indicators in the format of process components (data, process, output, and impact) [Table 6]. In the articles reviewed, different indicators were used in varied model applied for hospitals' evaluation, but a scoping review confirms this point that despite difference among performance evaluation models, common indicators are the main body of all reviewed models.

Table 6.

Effective indicators on hospital performance evaluation in a process format

Input	Process	Output	Impact
Number of inpatient beds	Number of hospitalization days	Number of deaths after admission	Patient satisfaction
Number of outpatient beds	Number of emergency visits to outpatient visits	Incidence of nosocomial infections	Relocation of staff
Number of physicians	Proportion of patients who have to use expensive medical equipment to total patients	Percentage of agreement between diagnosis at the time of admission and at discharge	Absence of staff
Number of nurses	Hospital survival rate	Number of outpatient deaths	Employee sick leave rates
Number of other clinical team staff	Combined index of hospitalization adjusted days	Number of deaths after surgery	Hospital success in obtaining credentials in quality management
Total number of employees equivalent to full time	Total number of nonemergency outpatient visits	Percentage of readmission for the same diagnosis	Legal complaint rate from the hospital during the year
Cost of human resources	Number of outpatient visits	Percentage of repeat surgical procedures	Staff satisfaction percentage
Cost of other resources	Number of emergency patients	The number of falling patients	Complaint patient percentage
Ratio of the number of administrative staff to total staff	Number of correct diagnoses to total diagnoses to each specialist	Percentage of medical errors
The ratio of physician to nurse	Number of patients refer to hospital to admitted patients	Percentage of postoperative hematomas and hemorrhages
Ratio of physicians to bed	Number of patients admitted per day	Mortality rate in intensive care unit
Ratio of nurses to bed	Number of minor surgeries	Needlestick
Ratio of other members of clinical team to bed	Number of major surgeries	Unplanned readmissions to the intensive care unit within 48 h after discharge
The ratio of the number of employees to the number of active beds	Patient admitted ratio	Prevalence of smoking among staff
Day-case rates	The ratio of perfect nursing documentation	Readmission rate per active bed
The ratio of staff to existing standards	Number of hours of nursing courses	Percentage of costs due to medical neglect
The ratio of managers’ salaries to the total cost of human resources	Ratio of appropriate prescriptions to total prescriptions	Error percentage in estimating bills correctly
Gross margin	Rate of clinical chemistry, hematology, immunology, and bacteriology tests	Number of readmissions based on different diagnoses
Clothing and appearance of staff	Average cost of outpatient	Hospital medical expenses relative to total hospital costs
Parking for patients’ relatives and acquaintances	Average cost of inpatient	Ratio of private income to total hospital costs
Prescription rate of diagnostic procedures	Average costs per bed per day	Sentinel event rate
Staff salary and benefits	Hand hygiene practices (measurement of alcohol)	Total rejected (total number of bills rejected by insurance)
Debtors	Percentage of patients with long stay	The rate of false-positive and false-negative tests
Quality of medical equipment	Average waiting time for outpatient admission	The rate of canceled surgeries
Hospital safety	Operating expense per bed	Total number of radiologic tests per number of beds
The reputation of the hospital	Duration of unused equipment	Medical error rates per 10,000 outpatients
Wellness facilities for patient families and visitors	Number of articles published in peer-reviewed scientific journals by staff per year	Compensation rate as a percentage of total revenue
On-call physicians	Occupied day per bed
Patient safety culture	Inpatient bed days per physician
Costs of staff training per capita	BOR
Percentage of staff with postgraduate degrees	BTR
Percentage of physicians’ referrals to other specialists	Waiting time for admitted in the emergency department
Ratio of total staff costs (salaries and benefits of staff) to total hospital revenue	ALS
Average overtime per employee	Waiting time for admitted in the emergency department
Ratio of assets to debt	ALS
Food quality	Time and circulation of visits
	The relevance of the tests prescribed to the diagnosed disease
	Physicians’ tolerance to hear details of patients’ problems
	Respectfully treat of all staff
	Number of discharge with personal consent

BOR=Bed occupancy rate, BTR=Bed turnover ratio, ALS=Average length of stay

Therefore, the researchers in this study tried to gather indicators that were mainly used in previous related literature to evaluate the performance of hospitals to guide the practice of those interested in this kind of topics.

Study novelty

In this study researchers tried to obtaining a simple but comprehensive list of indicators that commonly used for hospital performance evaluation instead of complex performance evaluation models.

Study limitation

Researchers in this study tried to extracted related articles but some of them were not available because of publisher or journal rules so, researchers for overcoming this limitation replaced the most similar ones.

Financial support and sponsorship

This article is the part of the first author thesis that financial by grant number IUMS/SHMIS-1395/9221532202

Conflicts of interest

There are no conflicts of interest.