Hospitals' efficiency in Iran: A systematic review and meta-analysis

Abstract

BACKGROUND:

Given that the need to pay attention to measuring efficiency is considered as one of the main pillars of improving the level of efficiency in hospitals, so this study was carried out aimed to determine the mean technical efficiency (The technical efficiency is bound by zero and one and a score of less than one means that the theatre is inefficient as it could) score in terms of type and activity of the hospital, input-oriented and output-oriented attitude, returns to scale (In economics, returns to scale and economies of scale are related but different concepts that describe what happens as the scale of production increases in the long run, when all input levels including physical capital usage are variable (chosen by the firm). The concept of returns to scale arises in the context of a firm's production function. It explains the behavior of the rate of increase) in hospitals of Iran using data envelopment analysis (DEA) (DEA is a nonparametric method in operations' research and economics for the estimation of production frontiers. It is used to empirically measure productive efficiency of decision-making units) and stochastic frontier analysis (SFA) (SFA is a method of economic modeling. It has its starting point in the stochastic production frontier models simultaneously introduced by Aigner, Lovell and Schmidt^[1977] and Meeusen and Van den Broeck^[1977]

MATERIALS AND METHODS:

The present study was carried out with a systematic review of all studies conducted on measuring efficiency of hospitals in Iran from March 21, 2001 to December 21, 2017 using DEA and SFA. Eleven databases were searched using appropriate keywords and 470 articles were found and evaluated using a checklist, and finally, 24 articles were entered into the meta-analysis process. Meta-analysis was performed using random effect model and fixed-effect model, and study heterogeneity was investigated using Q-Cochran test and I² index. Furthermore, the main reasons of study heterogeneity were identified due to meta-regression.

RESULTS:

The average technical efficiency score of hospitals using DEA and SFA method was obtained equal to 0.885 and 0.809, respectively. Furthermore, with regard to the DEA method, 0.885, 0.891.0.952 and 0.913 was obtained for input-oriented and output-oriented, general and specialized care hospitals and constant returns respectively. With regard to SFA method, 0.733, 0.664, 0.641, 0.802, was obtained, and the inputs and outputs affect measuring the efficiency.

DISCUSSION:

In contrast, the DEA method can investigate several input and output simultaneously and is used as an effective and flexible tool in order to measure the efficiency of the hospital. DEA can be easily used for calculating efficiency scores based on the proper selection of input and output indicators. The data envelopment analysis method and different input and output variables have been used in most studies conducted in Iran, and Stochastic Frontier Analysis has been less considered. In the present study, the DEA method in governmental educational hospitals showed a higher efficiency than SFA method in the hospitals under study. But in general, due to lack of optimal efficiency level in the hospital, it is suggested that policymakers determine the hospital efficiency indices in order to evaluate their efficiency from different dimensions.

CONCLUSION:

The average technical efficiency score of hospitals using DEA and SFA method was obtained equal to 0.885 and 0.809, respectively. Also, the mean technical efficiency score in terms of input-oriented and output-oriented, general and specialized care hospitals and constant returns to scale using the DEA method was obtained equal to 0.885, 0.891.0.952 and 0.913 and using the SFA method, respectively, it was equal to 0.733, 0.664, 0.641, 0.802, and the inputs and outputs affecting measuring the efficiency. There is no significant difference between the mean efficiency score between the two methods, but the data envelopment analysis method is used more. It is suggested that the hospitals efficiency indicators to be determined in order to more accurately evaluate the hospitals efficiency.

Introduction

Nowadays, in most developing countries, 5% to 10% of budget is dedicated to the healthcare sector. In the meantime, 50% to 80% of the health sector budget is dedicated to the hospitals. In Iran, about 40% of the total public health care costs are allocated to hospital care.[1] With the implementation of the Health System Development Plan, the share of health care costs from gross domestic product (GDP) (GDP is a monetary measure of the market value of all the final goods and services produced in a period of time, often annually or quarterly. Nominal GDP estimates are commonly used to determine the economic performance of a whole country or region and to make international comparisons) has increased. Such resources waste means that creating a certain level of output could be achieved using less resource.[2] On the one hand, the hospital uses a larger proportion of limited funding, and if it is organized in a no efficient manner, it leads to reduce the quality of life.[3] The presence of these challenges reinforces the incentives to create efficiency in hospitals[4] and encourages managers to identify inefficient hospitals and identify inefficiency sources[5] because the hospital efficiency can be increased without adding the factors of production by eliminating the factors involved in inefficiency.[6] Hence, if they want to show the efficiency in the organization, the first step is to analyze its efficiency and its basic condition is to apply a suitable method for measuring efficiency.[5] Today, there are different methods to measure hospital efficiency. Data envelopment analysis (DEA) and stochastic frontier analysis (SFA) methods are the most commonly used methods.[7] The SFA method compares the actual efficiency of hospitals using frontier estimation of the efficiency and indicates that frontier analysis methods have an important role in efficiency score estimation,[8] but in the DEA method, the selection of input and output variables have a significant effect on the results.[9] Furthermore, it is possible that selecting the input and output variables have a negative effect on individual and collective health. For example, the use of residence time index as an output may encourage hospitals to admit patients with a very simple and uncomplicated disease.[10] Therefore, it is necessary to identify the hospital input and outputs that are commonly used to measure hospital efficiency in an accurate way.[11] In the last few decades, measuring and analyzing efficiency has attracted the attention of researchers as one of the most effective measures in hospital research.[12] However, these studies provide an image of the hospitals' efficiency in a particular region; so, researchers use overview studies and combination of findings obtained from primary studies to provide a more comprehensive image of the efficiency of hospitals in a country.[13] Furthermore, selecting the most suitable set of input and output variables considered as a critical step in measuring efficiency.[5] No studies have been carried out on the same combination of these variables. Therefore, the present study has been carried out on investigating the systematic and meta-analytic study of measuring the efficiency of hospitals for Iran during 2001–2017 due to the breadth of research, the presence of different methods of measuring efficiency, not comparing them with each other, and finally, the use of different inputs and outputs to measure the efficiency of hospitals. After determining the mean technical efficiency score in terms of type, activity, selective model and ownership of the hospital in both DEA and SFA methods, they are analyzed using their meta-regression analysis, and input and output variables affecting optimal decision-making are introduced to the policymakers and the healthcare system managers of the country. We hope to take an effective step toward improving the hospitals' efficiency using the results of this study.

Materials and Methods

The present study was carried out using systematic and meta-analysis methods and includes all studies that measure the hospitals' efficiency in Iran using DEA and SFA methods.

Data collection method

Search strategy

In the present study, the electronic search of the subject has been done in published articles in domestic and foreign journals, theses, conferences available in Persian databases of SID, Iranmedex, Magiran, Medlib, Civilica, Irandoc and English databases of Web of Science, Pubmed, Scopus, Science Direct, Google Scholar, and the WHO site between March 21, 2001 and December 21, 2017, and the search strategy has been mainly performed based on Persian and English keywords using the Mesh system, with the possible combination of important, original, and sensitive words. The search has been done using Persian keywords of Efficiency, Hospital technical efficiency, DEA, SFA, efficiency improvement, and English keywords of DEA and SFA and Iranian hospital with And and OR Operators. Also, the list of reference in the studies published has been reviewed to increase sensitivity and select more numbers of studies.

Selection of studies

Of 470 articles, 344 studies were repetitive and removed. Then, 12 nonrelated studies were removed using exclusion criteria, such as studies that did not use the SFA and DEA methods to measure the efficiency level, the studies which have reported the level of efficiency qualitatively and the articles' letter to the editor. One hundred and fourteen articles were extracted for the study of full text, of which 26 articles were removed because of nonpublication of articles, the impossibility of contacting writers, and the limitations on the purchase of these articles. After determining the studies, to assess the quality of the documentation, a checklist was prepared; so that, the aim of each research, study method, sample size, sampling method, data collection tool, measurement status of variables, target group, and status of analysis were investigated using twelve questions (extracted from Cochran methodology). Finally, 24 articles entered the meta-analysis process (With reference to Figure 1).

Figure 1.

Flow diagram of literature search

Statistical analysis

Random effect model and fixed-effect model were used to perform meta-analysis and the study heterogeneity was investigated using Q-Cochran test and I² index. The agreement level between the two browsers (Cohen's kappa coefficient (κ) [Cohen's kappa coefficient is a statistic which measures inter-rater agreement for qualitative items. It is generally thought to be a more robust measure than simple percent agreement calculation, as κ takes into account the possibility of the agreement occurring by chance]) was determined using the Cohen test. The DerSimonian and Laird method was used to calculate the effects of community indicators (Pooled Effect Muser) for random models, and inverse variance method was used for fixed and forest plot models. The funnel plot was used to identify the publication bias qualitatively and Egger's regression test was used to identify the publication bias quantitatively. Furthermore, the main reasons for the study heterogeneity were identified using meta-regression. Meta-analysis was performed using the chart drawing with comprehensive meta-analysis V2 (CMAV2) software.

Results

According to Table 1 and based on most studies conducted in 2013, the DEA method is more investigated compared to SFA method to measure the efficiency of Iranian hospitals more than other cities. Also, in terms of 4 variables, input and output variables have been used to measure the efficiency. Four variables have been used as input and output variables for measuring efficiency.

Table 1.

Specifications of studies in meta-analysis and metaragiosis

No.of output variable	No.of Input variable	Standard error scale	Standard error management	Standard error technical	Mean scale efficiency	Mean management efficiency	Mean technical efficiency	No.of hospital	Estimation method	City studied	Publication year	Author(s)	No
2	4	0/025	0/01	0/032	0/9	0/97	0/87	19	DEA	Tehran	2015	Rezapur et al.[1]	1
4	4			0/026			0/826	54	DEA	Tehran	2016	Kakman et al.[14]	2
5	3			0/215			0/78	16	DEA	Tehran	2009	Alamtabriz et al.[5]	3
3	3			0/165			0/855	12	DEA	Yazd	2008	Mohamadi ardekani et al.[15]	4
4	4			0/052			0/965	12	DEA	Tehran	2009	Purreza et al.[16]	5
4	4			0/015			0/901	65	DEA	Iranian hospital	2012	Sepehrdoost et al.[17]	6
4	5			0/167			0/861	22	DEA	Tehran	2013	Adelazar et al.[18]	7
3	4			0/055			0/958	13	DEA	Yazd	2012	Askari et al.[19]	8
4	4			0/125			0/893	25	DEA	Tehran	2006	Ghaderi et al.[20]	9
4	4			0/082	0/98	0/961	0/943	19	DEA	Gilan	2013	Mohebifar et al.[21]	10
4	4			0/157			0/92	7	DEA	Kermanshah	2014	Ghasemi et al.[22]	11
2	3	0/169	0/056	0/165	0/89	0/95	0/85	12	DEA	kordestan	2016	Rezaei et al. .[23]	12
3	5	0/045	0/044	0/074	0/957	0/971	0/933	31	DEA	Iranian hospital	2016	Mehraban et al.[24]	13
5	6			0/193			0/593	12	SFA	Tehran	2013	Godarzi et al.[25]	14
1	4			0/184			0/68	12	SFA	kordestan	2016	Rezaei et al.[26]	15
1	4			0/173			0/624	9	SFA	kerman	2012	Godarzi et al.[3]	16
4	4			0/115			0/73	13	DEA	Lorestan	2014	Godarzi et al.[27]	17
4	4			0/099			0/95	13	SFA	Lorestan	2014	Godarzi et al.[27]	18
3	2			0/321			0/63	13	DEA	Tehran	2015	Khodabakhshi et al.[28]	19
2	2			0/013			0/88	11	DEA	Khorasan jonoobi	2013	kazemi et al.[29]	20
2	3			0/15			0/828	10	SFA	Tabriz	2015	Salmanibishak et al.[30]	21
2	4			0/229			0/389	19	SFA	Tehran	2015	Rezapur et al.[1]	22
2	4			0/135			0/622	7	SFA	Kermanshah	2015	Godarzi et al.[31]	23
2	4			0/099			0/839	64	SFA	Iranian hospital	2013	Hatam et al.[32]	24

Figure 2 shows the mean technical efficiency of hospitals' inverse variance by size, using DEA and SFA methods. In this figure, the efficiency estimation for DEA-based studies is closer to one, indicating a higher efficiency of these hospitals than hospitals which use the SFA method.

Figure 2.

Mean Technical Efficiency Of Hospitals’ inverse Variance By Size

Discussion

In the present study, 24 articles were reviewed using systematic, meta-analysis, and meta-regression methods. Of these, 16 articles using DEA method and 8 articles using the SFA method have measured the efficiency of governmental hospitals, and 3 articles have measured the efficiency of nongovernmental hospitals evaluated using DEA method Thirteen studies using DEA method and 2 studies using SFA have measured the efficiency of noneducational hospitals; 9 studies using DEA method and 5 studies using SFA method have measured the efficiency of specialized hospitals; 14 studies using DEA method and 7 studies using SFA method have measured the efficiency of general hospitals; 14 studies using input-driven approach and 2 studies using output-driven approach and 5 studies using the DEA method and 3 studies using SFA method with constant returns to scale approach have measured the efficiency.

In this study, four input variables including number of active beds, number of physicians, number of nurses, and other hospital staff, and four output variables including the number of admissions of outpatients and number of hospitalized patients, bed occupancy rate (The occupancy rate is calculated as the number of beds effectively occupied [bed-days] for curative care [HC.1 in SHA classification] divided by the number of beds available for curative care multiplied by 365 days, with the ratio multiplied by 100), and number of surgeries were identified as the most important and effective variables in measuring efficiency. Meta-analysis and meta-regression and the study of the number of variables have not been addressed in previous studies. For example, Mosadeghrad et al. during a study entitled “The efficiency of Iranian Hospitals: A Systematic Review and Meta-Analysis of Two Decades of Research,” in 2016, 91 articles which used DEA and SFA and Pabon Lasso methods during 2016–2017 to measure the efficiency of hospitals were evaluated and analyzed.[13] However, meta-analysis and meta-regression were not performed, but important variables such as the number of beds and hospital staff were identified as input variables, and the number of surgeries, the number of admissions of outpatients and number of hospitalized patients, and bed occupancy rate were identified as the most important output variables.

In 2016, Emanrezaei and Barun during a study entitled “Efficiency Analysis of Hospitals in Iran: A Systematic Review” investigated 18 articles that used DEA and SFA methods during 2006–2014 to measure hospital efficacy[33] and concluded that most of the studies have used the DEA method and the input-oriented approach, which is consistent with the present study and the study of Jahangiri entitled “Application of Data Envelopment Analysis Technique in Iranian Hospitals: A Systematic Review” in 2015, which was carried out by examining 47 articles (regardless of the time of publication).[34]

The input and output variables introduced by Jahangiri who have used DEA method in their study is similar to this study, but both methods were investigated by Emanrezaei and Baruni and the inputs affecting measuring efficiency included the number of full-time physicians, full-time nurses, and full-time staff, the number of active beds, the fixed bed, the annual cost, and the infrastructure and output variables included the number of hospital admissions and outpatients, the number of surgeries, bed occupancy rates, bed turnover rate (Hospital Bed Turnover Rate), (It is given by the formula: Hospital Bed turnover rate equals to Number of discharges (including deaths) in a given time period divided to Number of beds in the hospital during that time period), hospitalization day, the average length of stay in hospitals (ALOS), occupancy bed, hospital income, moreover, the number of discharged patients has been determined, by additional variables introduced by them, but as in the study carried out by Jahangiri, meta-analysis and meta-regression were not used in their study. Kiadaliri et al., in 2011–2012, in a study entitled “Stochastic Frontier Analysis in Measuring the Efficiency of Iranian Hospitals: A Systematic Review and meta-regression Analysis,” examines both the DEA and SFA methods and its meta-analysis and meta-regression estimated the mean efficiency of hospitals at 0.846.[35] In addition, as in this study, they concluded that the most studies in the field of measuring efficiency were carried out in 2012.

A number of systematic reviews were carried out on measuring efficiency of healthcare centers around the world, and inputs and outputs have been proposed. For example, a study entitled “A systematic review of hospital input and output in measuring technical efficiency using DEA” was carried out by Azreena and Rosliza in 2018, in which the input and output used in measuring efficiency were investigated, and the best indicators were introduced. Input and outputs which are usually used by researchers for the analysis of the technical efficiency of a hospital using DEA include the number of physicians, the number of nurses and the number of beds, the number of other nonmedical staff and the total number of employees, the total cost, the total cost of nonstaff, the value of fixed capital, and the cost of drug storage and outputs include the total number of admissions, mean daily admission, number of outpatients, number of surgeries, number of deliveries, ALOS, bed occupancy rate, and total income. Although the present study identified input and outputs, the researchers believe that the selection of data and outputs should be defined according to the goals of each hospital.[11] More input indicators are proposed compared to this study.

Hussey et al. during a study entitled “A systematic review of measuring the health care efficiency” investigated 172 English-language articles from 1990 to 2008 and compared two methods of DEA and SFA in measuring the efficiency of the health system and investigated the number and type of selected input and outcomes. In the present study, the days of discharge, visits of doctors, results of health measures, and therapeutic procedures were investigated as outputs.[7] Katharakis and Katostaras, during a study entitled “SFA, DEA for Measuring Health Care efficiency: A Systematic Review,” investigated 21 English-language articles that had been accepted by various journals over the past decade. In these articles, the inputs were analyzed using meta-analysis and the relationship between input and outputs was investigated by measuring more accurately the efficiency, better decision-making, their effectiveness on the efficiency, and selecting an appropriate model to measure efficiency and proved that the accuracy of both methods depends on many factors, including statistical methods, definition of inputs and outputs, and access to data.[36] Dong et al., during a study entitled “Measuring the efficiency of Chinese hospitals using data envelopment analysis for: a systematic review,” concluded that significant numbers of input and output were used in China, Europe, the United States, and other countries.

On the other hand, there are some shortcomings such as inappropriate selection of input and output indices and unmodified errors on efficiency score in the studies conducted on the efficiency of hospitals based on DEA. Therefore, it is necessary to investigate and select an appropriate index for using the DEA method.[37] Binder and Rudolph, in another study entitled “The systematic review of health care centers efficiency measurement” reviewed 21 studies and concluded that efficiency measurement of healthcare organizations has direct effect on their policy. They found that the difference in efficiency determined by DEA and SFA methods is due to various factors such as statistical errors, input and outputs definitions, and available data. However, the views of the different models have different advantages and disadvantages, and selecting the most appropriate method depends on the type of organization under study and available data.[38]

Hofmarcher, during a study entitled “Measuring Australian Hospitals Efficiency Using the DEA Approach,” concluded that in most studies, the number of beds is considered as a input variable and the number of hospital staff, the rate of discharge, the length of stay, and emergency visits are considered as output variables, but what is considered as a DEA advantage compared to other methods is that several inputs and several outputs are arisen in the process of efficiency measurement; but, on the other hand, the selection of the most suitable inputs and outputs is concern of researchers in this field of studies.[39]

Chirikos and Sear during another study entitled “Measuring Hospital Efficiency: Comparing Two DEA, SFA approaches” compared two methods and concluded that significant differences were observed over the years between the DEA and SFA approaches although there is no clear relationship between the characteristics of different hospitals and DEA scores with SFA scores; but after comparing these two methods, it has been concluded that different results are obtained due to the existence of different methodologies and inputs in calculating the efficiency using the two methods.[40] According to the results of comparing the studies carried out in abroad, we conclude that the researchers agreed on the inputs and outputs defined in the present study as the most effective indicators in calculating efficiency and emphasized that input and output play an important role in the efficiency rate, and it cannot be said decisively that a method is better than another. Of course, DEA and SFA are widely used in efficiency measurement. The SFA method is often used for analysis of no efficient hospitals although this method has clear and important advantages; but, the disadvantages of this method for hospital efficiency measurement are also evident such as the need for production functions estimation and the use of an output.

In contrast, the DEA method can investigate several input and output simultaneously and is used as an effective and flexible tool to measure the efficiency of the hospital. DEA can be easily used for calculating efficiency scores based on the proper selection of input and output indicators.[37] The DEA method and different input and output variables have been used in most studies conducted in Iran, and SFA has been less considered. In the present study, the DEA method in governmental educational hospitals showed a higher efficiency than SFA method in the hospitals under study. However, in general, due to lack of optimal efficiency level in the hospital, it is suggested that policymakers determine the hospital efficiency indices to evaluate their efficiency from different dimensions.

Conclusion

The average technical efficiency score of hospitals using DEA and SFA method was obtained equal to 0.885 and 0.809, respectively. Also, the mean technical efficiency score in terms of input-oriented and output-oriented, general and specialized care hospitals and constant returns to scale using the DEA method was obtained equal to 0.885, 0.891.0.952 and 0.913 and using the SFA method, respectively, it was equal to 0.733, 0.664, 0.641, 0.802, and the inputs and outputs affecting measuring the efficiency. There is no significant difference between the mean efficiency score between the two methods, but the data envelopment analysis method is used more. It is suggested that the hospitals efficiency indicators to be determined in order to more accurately evaluate the hospitals efficiency.

Research constraints

The lack of access to a number of articles due to access restrictions on the university site is considered as one of the constraints in this study that the websites of other universities were used to solve this problem. Furthermore, it is suggested that an appropriate method for efficiency measurement using systematic review and meta-analysis to be identified in future studies aimed to the optimal allocation of resources in Iranian hospitals.

Financial support and sponsorship

This research was supported by the Vice-Chancellor of Research of Shahid Sadoughi University of Medical Sciences in Yazd.

Conflicts of interest

The article has not been registered elsewhere before and will not be published.