Performance evaluation of Al-Zahra academic medical center based on Iran balanced scorecard model

Abstract

Background:

Growth and development in any country's national health system, without an efficient evaluation system, lacks the basic concepts and tools necessary for fulfilling the system's goals. The balanced scorecard (BSC) is a technique widely used to measure the performance of an organization. The basic core of the BSC is guided by the organization's vision and strategies, which are the bases for the formation of four perspectives of BSC. The goal of this research is the performance evaluation of Al-Zahra Academic Medical Center in Isfahan University of Medical Sciences, based on Iran BSC model.

Materials and Methods:

This is a combination (quantitative–qualitative) research which was done at Al-Zahra Academic Medical Center in Isfahan University of Medical Sciences in 2011. The research populations were hospital managers at different levels. Sampling method was purposive sampling in which the key informed personnel participated in determining the performance indicators of hospital as the BSC team members in focused discussion groups. After determining the conceptual elements in focused discussion groups, the performance objectives (targets) and indicators of hospital were determined and sorted in perspectives by the group discussion participants. Following that, the performance indicators were calculated by the experts according to the predetermined objectives; then, the score of each indicator and the mean score of each perspective were calculated.

Results:

Research findings included development of the organizational mission, vision, values, objectives, and strategies. The strategies agreed upon by the participants in the focus discussion group included five strategies, which were customer satisfaction, continuous quality improvement, development of human resources, supporting innovation, expansion of services and improving the productivity. Research participants also agreed upon four perspectives for the Al-Zahra hospital BSC. In the patients and community perspective (customer), two objectives and three indicators were agreed upon, with a mean score of 75.9%. In the internal process perspective, 4 objectives and 14 indicators were agreed upon, with a mean score of 79.37%. In the learning and growth perspective, four objectives and eight indicators were agreed upon, with a mean score of 81.11%. Finally, in the financial perspective, two objectives and five indicators were agreed upon, with a mean score of 67.15%.

Conclusion:

One way to create demand for hospital services is performance evaluation by paying close attention to all BSC perspectives, especially the non-financial perspectives such as customers and internal processes perspectives. In this study, the BSC showed the differences in performance level of the organization in different perspectives, which would assist the hospital managers improve their performance indicators. The learning and growth perspective obtained the highest score, and the financial perspective obtained the least score. Since the learning and growth perspective acts as a base for all other perspectives and they depend on it, hospitals must continuously improve the service processes and the quality of services by educating staff and updating their policies and procedures. This can increase customer satisfaction and productivity and finally improve the BSC in financial perspective.

INTRODUCTION

Hospitals in healthcare system are confronted with a wide range of external pressures. These include societal changes such as demographic and epidemiological changes, increased expectations of communities and patients, mobility of professionals, and difficulties with recruitment and retention of staff, increasing cultural diversity, and rising financial concerns.[1,2] Healthcare is so important that it should never be affected by shortages and economic and financial savings. This means that whenever a community is faced with more economic and financial resource shortages, serious attention should be paid to healthcare status in that community. The opportunity to address the special social and economic development in various categories provides for scientific and practical challenges to overcome. It is fair to say that the level of development in a country depends upon the level of health in that country, and it is the basis of development in other fields.[3]

Hospitals operate in an environment demanding demonstration of excellent care in a challenging financial environment where markets for such services are highly competitive. This requires attention to be paid to multiple aspects of clinical, financial, and operational performances.[4]

For understanding the desirability and quality of services, especially in complex and dynamic environments such as healthcare systems, there is a need for a systematic evaluation of those services. The lack of assessment and control system means lack of communication with organization's internal and external environment, which leads to the lack of efficiency, decreased effectiveness, and organizational death.[5]

In the past, the traditional method of performance evaluation of the organizations has been financial assessment. The financial indicators that traditionally are evaluated in most organizations described only the past events, which is not useful and adequate for organization strategic planning. Paying attention to the limitations mentioned above, the traditional methods of performance evaluation that were based on financial indicators not only have do not have sufficient qualifications in determination of reasons for success or failure of the organization, but there is not a logical and causal relationship between the factors which resulted in success or failure. Hence, it was unable to support the management programs, especially the organization strategic programs.[6–8] In order to cope with the challenging nature of running a hospital, healthcare managers are implementing a new professional performance management tool called “The Balanced Scorecard (BSC).” The BSC was first introduced in the early 1990s through the work of Harvard Professor, Dr. Kaplan , and Norton. Since then, the concept has become well known and its various forms widely adopted across the world.[9] The BSC is a technique that has been used in order to do performance evaluation in many organizations, including health settings, hospitals, and physician practices to put strategy at the center of the organization by translating strategy into operational objectives.[10,11] The BSC traditionally includes both tangible and intangible measures divided into four major perspectives.[12,13]

In the financial perspective, indicators of financial performance are defined as the long-term objectives of organizations, which also specify whether or not the implementation of a strategy is effective in improving the profitability of the organization. A financial control system that is designed well can actually improve and enhance the comprehensive quality management of an organization.

In the customer perspective, in order to select the objectives and indicators for a customer's perspective, the organizations must respond to two critical questions: first, who are our target customers? And second, what are our proposed values for them? Michael Porter believed that lack of focus on customer's specific demands and their desired values causes the organization not to achieve its competitive advantage.

In the internal process perspective, the organizations must determine some processes, which by improving them can continue to make value for their customers. Achievement of each one of the objectives determined in the customer's perspective is dependent upon how efficiently and effectively the internal operating processes are fulfilled.

In the learning and growth perspective, all organizations maintain their value by using the subjective, such as human capital, information systems, high-quality processes, capability to respond to requirements, relationship with customers, creative abilities, and organizational culture. The response to the question of how an organization can fulfill the objectives of the internal process perspective and the customer perspective lies in the fulfillment of the objectives and indicators of learning and growth perspective. Fulfillment of these objectives and indicators are enabling the factors for the accomplishment of the objectives and indicators in the three other perspectives. In fact, they are the infrastructure for the implementation and the establishment of the BSC.

The BSC can help organizations to “balance” factors including short-term and long-term objectives of the organization, the financial and non-financial indicators of success, the lag and lead indicators of performance, and the internal and external constituents of the organization.[14,15]

There are many different BSC applications in all types of industries, both in the United States and internationally. Several articles and books have been written on the BSC methodology and there are a variety of software products to assist implementation of this performance measurement technique. Historically, performance improvement systems have focused on measurements and indicators alone. What is unique about the BSC approach, in contrast to other methods, is that it links strategy with four performance perspectives and goes beyond the traditional financial measurements in determining whether or not an organization has been successful. Integral to BSC is the notion that an organization's strategies and their execution are among the most important factors in performance improvement.[5,14,15]

Healthcare organizations have had to meet some unique challenges in adopting the BSC to healthcare settings. Since 1994, when the first article was published on BSC in healthcare organizations, different articles have been written about BSC in the health services and management literature, making the BSC appear to have gone into a growth phase. According to Zelman et al.'s study, the BSC has been adopted by a broad range of healthcare settings, including hospital systems, psychiatric centers, and national healthcare organizations.[16]

Many different researches have been done in Iran about implementation of BSC. A research done by Vatanhah and Salemi, surveying the evaluation system in Iran University of Medical Sciences affiliated hospitals using BSC, showed low scores in all perspectives. Begloo in his research surveyed the BSC model in selected countries and Iranian public hospitals to design a model for hospitals in Iran.[17] This research was conducted with the aim of evaluating the performance of Al-Zahra Academic Medical Center at Isfahan University of Medical Sciences, by using the Begloo BSC model for comparison.

MATERIALS AND METHODS

The research is a combination (quantitative–qualitative) research which has been done in Al-Zahra Academic Medical Center at Isfahan University of Medical Sciences in 2011. The research populations were hospital managers at different levels. Sampling method was purposive sampling in which the key informed personnel participated in determining the performance indicators of hospital as the participants of the BSC team at focused group discussion (FGD). After determining the conceptual elements in FGD, the performance objectives (targets) and indicators of hospital were determined and sorted in perspectives by the group discussion participates. Following this, the performance indicators were calculated by the participants of the BSC team according to the predetermined objectives; then, the score of each indicator and the mean score of each perspective were calculated. In qualitative part of research, the data were collected by conducting 12 FGDs by the participants of the BSC team using a checklist. The construct and content validity of the checklist were confirmed several times by reviews and with the help of research supervisors, consultant and senior hospital administrators, and the Iran BSC model. In the quantitative section, performance indicators were calculated using descriptive statistical methods.

RESULTS

In this research, the participants through several sessions of FGD specified five strategies derived from mission and vision of the hospital. Strategies included customer satisfaction, continuous quality improvement, human resource development, supporting innovation, and improving productivity. The participants also specified four perspectives for the BSC of Al-Zahra Academic Medical Center, which are patients and community perspectives, internal processes perspectives, learning and growth perspective, and financial perspective. For each perspective, they also specified several objectives or targets in order to facilitate fulfillment of the hospital strategies.

Increasing satisfaction of the service recipients and the level of responsiveness to the customers were considered as the targets of patients and community perspective by the research participants. Documentation of conceptual and functional elements and entities, development of appropriate activities for patient's needs and community, improvement of service quality level, and time management were considered as the targets for internal processes perspective by the study participants. Targets of learning and growth perspective approved by participants included human resources development, employee's motivation, improving hospital information technology, supporting innovation, and applied research. Financial perspective targets approved by participants were increasing hospital revenues and reasonable reduction of hospital's costs. These findings are illustrated in Table 1.

Table 1.

Strategies and targets in balanced scorecard of Al-Zahra academic medical center

Hospital performance indicators were developed during several steps using a checklist conforming to developed strategies and hospital capabilities. Thirty indicators were agreed upon, of which 3 indicators were classified in patients and society perspective, 14 indicators in internal process perspective, 8 indicators in learning and growth perspective, and 5 indicators in financial perspective of the BSC.

Using hospital data, the indicators for each perspective were calculated with the help of study participants and the score of each target was determined. By adding the score of indicators of each perspective, the final score of the perspectives in BSC of the hospital was determined and responded to the research questions. The results are illustrated in Table 2.

Table 2.

Score of the perspectives in balanced scorecard of Al-Zahra academic medical center

DISCUSSION

The BSC is a tool for measuring organizational performance in different perspectives. However, before it is implemented, one should review the organization conceptual elements such as mission, vision, values, objectives, and strategies. In this study, the participants of BSC FGD at Al-Zahra Medical Center first revised and formulated the conceptual elements of the hospital. Then, objectives and indicators of the hospital BSC were agreed upon in order to fulfill hospital vision and strategies. Finally, by exact calculation of the indicators and predetermined targets, the average score of indicators and the score of each perspective were specified, and thus answered the research questions.

Since the compilation of hospital strategies consistent with hospital mission and vision is an important step for implementation of the BSC, the BSC team participants of Al-Zahra Academic Medical Center agreed upon five strategies from the list of the present research and Iran BSC model strategies through the FGD. The strategies agreed upon are customer satisfaction, continuous quality improvement, human resources development, supporting innovation, and improving productivity. Although these strategies are different in description compared to Iran BSC model strategies, they are mostly similar in concept and have conceptual compliance with Iran BSC model. For example, customer satisfaction was accepted as one of the main strategies in accordance with hospital mission and vision, while in Iran BSC model was not included. In all the hospitals studied by Begloo, their strategies were offering high-quality services, innovation, and development of services, which are similar to the present research findings.

Selection of the perspectives for BSC model shall be done based on whatever the organization strategies are, which creates a competitive advantage.[18] By paying attention to this important point, participants in the FGD agreed upon four perspectives of patients and society (customer), internal processes, learning and growth, and financial perspective in BSC of the hospital. These perspectives have conceptual compliance with the perspectives of BSC model of Iran and the main perspectives of Kaplan and Norton model. Chen et al., accomplished a research titled “Using the balanced scorecard to measure Chinese and Japanese hospital performance,” determined four main perspectives in their scorecard, which are similar to the present research results.[19] In another research, some countries used different perspectives or added another perspective in accordance with their strategy, for example, Peters et al. used capacity for services provision perspective and overall vision perspective, reported in their research article titled “Measuring and managing progress in the Afghanistan health sector balanced Scorecard.” Their results are different from the findings in this research.[20] Urrutia and Eriksen in their results used the environment perspective as the fifth perspective of BSC of their hospital, which is different from the present research findings.[21]

In the present research, participants at the FGD determined 12 objectives based on hospital strategies, organizational structure, and scope of services. In Iran BSC model, the number of these objectives is 16, of which some are similar to the determined objectives in the scorecard of Al-Zahra center and some are different. Also, two objectives of increasing customers’ satisfaction level and increasing responsiveness level were agreed in the patient and society perspective of Al-Zahra center. Increasing responsiveness is one of the important objectives of any health system, which was introduced by WHO in order to increase satisfaction of patients and decrease their complaint. Improving the responsiveness in health systems is an intrinsic goal of health policy responding to patient's non-clinical needs. Responsiveness focuses on the interpersonal and contextual aspects of people's interaction with the health setting. For measurement purposes, responsiveness has been defined on eight domains: dignity, autonomy, confidentiality of information, communication of information, prompt attention, quality of basic amenities, access to support, and choice (of healthcare provider), which were not considered in Iran model.[22,23]

In the internal processes perspective of the present research, four objectives were agreed upon. The most important of these objectives was documentation of conceptual and clinical elements and entities which are used as a utilization tool for organization's activities. Documentations of conceptual elements, including establishment of clinical policies and procedures, clinical guidelines, nursing care plans, and clinical pathways, are the infrastructure necessary for the fulfillment of internal processes perspective.[24]

A study performed by Urrutia and Eriksen in Benito Menni Hospital at Spain on the development of the clinical policy and procedures is considered as one of the vital success factors for promotion of quality of services, which represent the importance of clinical conceptual elements.[21]

In the Iran BSC model, improving service quality was selected as one of the objectives of patient and community perspective. In our research, the participants selected the internal processes, based on the direct effect of internal processes on promotion of quality of services, which is similar to the research findings of Barati et al.'s study.[25]

In the present research, four objectives were determined in the learning and growth perspective, which has general conceptual compliance with Iran BSC model but differs in the number of objectives.

In the financial perspective, two objectives were determined by the participants, i.e. increasing hospital revenues and reasonable reduction of hospital's costs, which are similar to Iran BSC model. Increasing revenues and decreasing expenses are among the objectives of financial perspective of all hospitals of the Begloo research. Chen et al. and Urrutia and Eriksen also indicated increasing revenue as the objective of financial perspective in their researches, which are similar to the findings of our research.[19,21]

In the Iran BSC model, the number of indicators was 32. Some of them were not agreed upon by the participants at FGD in our research, considering the Al-Zahra hospital strategies and objectives, and some of them were finalized as the indicators of Al-Zahra hospital, with little or no change.

In the present research, the indicators such as percent of responsiveness to the patients, average hours of the internet use, average waiting time from the first triage until the first physician's visit in emergency room, average waiting time from the first physician's visit until the implementation of first nursing care plan, rate of net hospital mortality, hospital nosocomial infection rates, percent of canceled surgeries, number of discharge patients against medical advice, rate of compliance with the prophylactic antibiotic before surgery, percent of cesarean sections, and the percent of hospital deductions in FGD were agreed upon which are different from Iran BSC model.

In the customer perspective of the Ontario hospitals’ BSC model, the communication with patient and responsiveness to patient are the chosen indicators.[26] Also, in Griffith et al.'s study, the indicator of average length of patient stay that is one of their internal process perspective indicators is compatible with the findings of our study.[27]

Gao and Gurd in their research article titled “Managing with the Scorecard in not-for-profit healthcare setting” considered the number of accepted papers as one of the indicators of learning and growth perspective, which is also similar to our study.[28] Niven suggested that the needed time for access to the required information about organization activities is considered as the indicator of learning and growth perspective.[18] Amount of hospital's revenue is also one of the indicators of most countries which applied BSC as a tool for evaluation of hospital performance. Also, in a survey of the hospitals in Iran, the total hospital revenue is used as one of the financial indicators of their performance evaluation model.[17]

After finalizing the indicators in the BSC of Al-Zahra Medical Center, these indicators were calculated by participants at the FGD and scored against the projected targets. Then, the score of each perspective was determined using the average score of its indicators.

In the patients and society perspective, participants of BSC team at FGDs agreed upon two objectives and three indicators. Calculation of these indicators and their score against the specified targets was done, and also, the average score of this perspective was determined (75.9%).

In the internal processes perspective, participants agreed upon 4 objectives and 14 indicators. Calculation of these indicators and their score against the specified targets was done, and also, the average score of this perspective was determined (79.37%).

In the learning and growth perspective, participants agreed upon four objectives and eight indicators. Calculation of these indicators and their score against the specified targets was done, and also, the average score of this perspective was determined (81.11%).

In the financial perspective, participants of BSC team agreed on two objectives and five indicators. By calculation of these indicators and their score against the specified targets and also taking the average, the score of this perspective was determined (67.15%).

In this research, the learning and growth perspective gained the highest score and financial perspective gained the least score. The highest score in learning and growth perspective was because of the increased awareness level and performance of the staff, increased educational hours achieved by regular and periodic workshops, and committee sessions’ feedback results. The low score in the financial perspective is because of mismanagement and lack of appropriate method of resource allocation, expenditure of revenue, and utilization of resources.

In a research done by Chen et al. focusing on hospitals of Japan and China, improving the internal management has been introduced as a factor for improving efficiency and increasing the financial position which has similar condition to our hospital.[19] In a research conducted by Barati et al., the learning and growth perspective obtained the highest scores and financial perspective obtained the least score; the reasons were increased knowledge level of staff by increasing learning hours and improving staff education, and in their financial perspective, by not paying one due to poor performance evaluation and payment methods.[25]

In patients and society perspective of Al-Zahra center, management and staff tried to increase the percent of responsiveness and customer satisfaction and decrease the percent of customer complaints. In the Al-Zahra center's internal process perspective, the management and staff tried to improve documentation and establish policy and procedure in the area of their responsibility and also established and developed the conceptual elements at clinical and functional levels.

CONCLUSION

One way to create demand for hospital services is performance evaluation by paying close attention to all BSC perspectives, especially the non-financial perspectives such as customers and internal processes perspectives. The performance evaluation system must help to improve the efficiency and effectiveness in the hospital. The purpose of performance evaluation of a hospital is to identify opportunities that can improve the performance. The use of the quality models such as BSC can help in continuous monitoring and annual assessment of organization. Holding regular FGD systematically is also effective for logical approach to develop a strategic plan and to better understand and participate in group discussions, with the four perspectives of the BSC model in mind.[29–32]

The main core of hospital BSC consists of the conceptual elements of every organization and maintaining a balance in the perspectives’ results for the improvement of the organizational performance and an increase in the efficiency of the organization. Therefore, the learning and growth perspective is the base, and all other perspectives are built on it. Hospital must improve the processes and develop the quality of staff services by educating them and developing procedures and policies. This can increase customer satisfaction and finally promote productivity and improve the BSC in the financial perspective.

In general, the results of this study represent the fact that the BSC model can illustrate the rate of performance in each one of the BSC perspectives maintaining a balance between tangible and intangible assets of the hospital and the flexibility of the organizational structure in achieving its goals and targets as a result of selected set of indicators related to the main strategies.