Application of Analytical Hierarchy Process Approach for Service Quality Evaluation in Radiology Departments: A Cross-Sectional Study

Abstract

Background:

Radiology department as a service provider organization requires realization of quality concept concerning service provisioning knowledge, satisfaction and all issues relating to the customer as well as quality assurance and improvement issues. At present, radiology departments in hospitals are regarded as income generating units and they should continuously seek performance improvement so that they can survive in the changing and competitive environment of the health care sector.

Objectives:

The aim of this study was to propose a method for ranking of radiology departments in selected hospitals of Tehran city using analytical hierarchical process (AHP) and quality evaluation of their service in 2015.

Materials and Methods:

This study was an applied and cross-sectional study, carried out in radiology departments of 6 Tehran educational hospitals in 2015. The hospitals were selected using non-probability and purposeful method. Data gathering was performed using customized joint commission international (JCI) standards. Expert Choice 10.0 software was used for data analysis. AHP method was used for prioritization.

Results:

“Management and empowerment of human resources’’ (weight = 0.465) and “requirements and facilities” (weight = 0.139) were of highest and lowest significance respectively in the overall ranking of the hospitals. MS (weight = 0.316), MD (weight = 0.259), AT (weight = 0.14), TS (weight = 0.108), MO (weight = 0.095), and LH (0.082) achieved the first to sixth rankings respectively.

Conclusion:

The use of AHP method can be promising for fostering the evaluation method and subsequently promotion of the efficiency and effectiveness of the radiology departments. The present model can fill in the gap in the accreditation system of the country’s hospitals in respect with ranking and comparing them considering the significance and value of each individual criteria and standard. Accordingly, it can predict an integration of qualitative and quantitative criteria involved and thereby take a decisive step towards further efficiency and effectiveness of the health care evaluation systems.

1. Background

Service quality is an important element in growth, success and survival of any given organization, assuming the importance of a principle for predicting future developments of the organization (1).‎‏‏‏‎‎‏ Considering the ongoing globalization process, health care rendering systems’ development as well as the increase in people’s awareness, maintaining sustainable service quality accessible to all is of most importance to the organizations (2, 3). In today’s competitive environment, health care providers have forced to make breakthrough, control cost and increase quality (4-7). Health care provider organizations globally have focused on the use of clinical quality indexes to identify health care improvement opportunities, measurement of dedicated interventions efficiency and presenting a quantitative link between care quality and cost effectiveness (8). ‎‏Quality plays a major role in customer satisfaction as recipients of services and products. Thus, all organizations interested in accurate and reliable tools to assess the quality of services (9, 10).

To make a conscious decision regarding hospital service quality, accurate and reliable information is needed, especially for care systems whose objective is to minimize the costs (11). Wollmann et al. suggest that evaluation of health care services is an important element in defining a suitable and cost effective health care system (12). Accreditation is a constant tool that promotes the desirable standards and improves the results in the health care sector (9, 10, 13, 14). In public sector systems, it is considered as a quality assurance solution (15). Accreditation is not a fast modification program, but rather a long term strategy (16).‏‎‎ Accreditation program having been implemented in many developing countries, is used as a monitoring tool that will assure service quality and optimum resource use (13). ‎Haj-Ali et al. (14) studied the relationship between hospital accreditation and patient’s satisfaction through evaluation of patient’s willingness to recommend the hospital to others. ‏‎‎‏‏

Accreditation program is renowned as a comprehensive accreditation program in the world by joint commission international (JCI) standards (17). Undertaken studies indicate that JCI standards have provided a specific framework and systematic methodology for efficiency and effectiveness of the hospital (18). Moreover, numerous studies have been carried out using JCI standards for evaluation of the health care centers. Amerioun et al. (19) in the laboratory of a military hospital and Bahadori et al. (20) for assessment of selected military hospitals used JCI standards.

The analytical hierarchical process (AHP) method formulates decision making for problems in a hierarchical structure and therefore resolves complicated problems in areas such as planning, resource evaluation, efficiency measurement, resource allocation, selecting the best policy after finding a set of options, and prioritization adjustment. It provides a suitable analyzing method because AHP is a multi-criteria decision making technique that allows subjective as well as objective parameters to be considered in the decision making process. The hierarchical process provides the possibility of studying the problem as a whole, while paying attention to the interaction between intra-hierarchical components. In this method, each decision making problem can be designed in a tree framework, so that the levels of the tree include the goals, criteria to achieve the goals, sub-criteria and finally, the understudy options. By breaking the problem into decision-making levels, it can focus on the smaller set of decisions (21).

Diagnostic radiology is an integral part of the health care system with many clinical advantages influencing the significant decisions in any of the patient management procedures. At the same time, in case the quality of the services provided is lower than the desirable level, unnecessary costs shall arise from the consequences, putting the health care system of the country under pressure. Therefore, evaluation of this department has gained considerable importance (3). ‎Investigating the present status and comparing it with the valid standards is considered as the key factor in reliability and usefulness of this facility (22). Measuring the service quality and ranking of the hospitals has the potential that enables health care investors to identify weak performances. Meanwhile, the patients and the consumers can choose the best hospital to solve their problem. The hospitals can improve their performance (23). Results of this program are of high importance, especially when considering their vast consequences specifically on the budget and financial allocations of the hospitals (20). This study is the first study that has been conducted on evaluating the quality of radiology services using AHP. The main objective of the present study was to introduce and evaluate AHP method as a good model for ranking radiology departments based on the provided services.

2. Objectives

The main objective of this study was to propose a ranking method for radiology departments of selected understudy hospitals in Tehran using AHP and quality evaluation of their services.

3. Materials and Methods

3.1. Study Design

This study was an applied and cross-sectional study.

3.2. Setting

The study was carried out in radiology departments of hospitals affiliated to Shahid Beheshti university of medical sciences, Iran in January of 2015. Based on the taxonomy of hospitals, only homogenous hospitals could be compared with each other, the hospitals participated in this study were similar in structure; i.e. they were educational, and grade 1 (based on the grade granted to them by the Ministry of Health and Medical Education) hospitals and the average weight of referees to the department of radiology in these hospitals were close to each other. Also, the number of beds and consequently the number of services provided by them were almost close to each other.

3.3. Samples

In this study, six hospitals affiliated to Shahid Beheshti university of medical sciences were selected. The hospitals were selected using non-probability and purposeful method. Participants in this study consisted of 10 evaluation experts of the hospitals who collaborated with the third author in the process of weighting and ranking hospitals based on JCI standards. At least 5 years of working experience in hospital evaluation was considered as inclusion criteria for selecting the evaluators. They were selected using purposeful sampling.

3.4. Data Sources/Measurement

3.4.1. First Phase

In this study, the localized JCI standard checklist was used. The checklist includes standards oriented around four axes of management and organizing, management and empowerment of human resources, safety, quality improvement and data accumulation requirements and facilities.

3.4.2. Second Phase: Multi-Criteria Decision Making Using AHP

To study the examined criteria weights, a researcher-made questionnaire was prepared using JCI standards and designed in matrix of paired comparisons. The source of the questionnaire was JCI standards, which was a standard source. In this questionnaire, measures (criteria) were first compared two by two; then the hospitals were compared two by two based on each criterion. Six questions were designed for weighting the criteria; and 15 questions for weighting each hospital based on each criterion. As there were four criteria, a total of 60 questions were designed based on four criteria. For content validity of the questionnaire, the viewpoint of experts working in the scope of hospital evaluation, as well as the indices of the content validity index (CVI) and content validity rate (CVR), which were equal to 0.78 and 0.77, respectively, were used. Cronbach’s alpha coefficient was used for reliability, which was equal to 0.88. The importance of each criterion compared with other criteria was determined using numbers 1 - 9 (Table 1). In this study, the criteria that were the same as JCI standards were first compared two by two by the experts using a matrix of pairwise comparisons. The weight and ranking of criteria (JCI standards) were determined based on this comparison. After this stage, the hospitals were compared with each other, two by two, using a matrix of pairwise comparisons and based on main topics of JCI, the rank and weight of each hospital in each topic was generally determined.

Table 1. Importance of Each Criterion Relative to Other Criteria.

Comparison Values	Importance
1	Equal importance
3	Moderately important
5	Strongly important
7	Very strongly important
9	Extremely important

3.5. Data Analysis

Prioritization was done using AHP, which is one of the frequently used methods for multi criterion decision making (MCDM) using expert choice 10.0 software. AHP was developed by Thomas L. Saaty in the 1970s. This method analyzes the issues like the analyses performed in the human brain. AHP enables the decision-makers to determine the simultaneous interactions of many complex and uncertain situations and helps them set the priorities.

3.6. AHP Steps

- Drawing the hierarchical tree: In the hierarchical tree, the purpose of the study was placed at the first level (selection of the best department of radiology). JCI standards and options (radiology departments) were placed at the second and third levels, respectively (Figure 1).

Figure 1. ‎The hierarchical tree of criteria weight evaluation.

• - Performing pairwise comparisons: The matrix of paired comparison is formed at this level. Then paired comparisons are conducted by the experts using a scale of 1 to 9 to determine the relative importance of each option compared to the other options using any criteria.

• - Normalizing pairwise comparisons matrix: In order to normalize each criterion in each column of the matrix, each number is divided by the sum of the numbers of that column.

• - Calculating relative weights: To calculate the relative weights of each criterion, the arithmetic mean of each row is calculated. It means that the sum of the numbers in each row of the matrix is divided by the total number of the numbers in that row.

• - Multiplying relative weights of the criteria by relative weights of the options: After calculation of the relative weights of the criteria and the relative weight of the options, they are multiplied together.

• - Ranking the options: Values obtained by multiplying the criteria and options are sorted based on the order of value; the options are prioritized.

• - Calculating inconsistency rate: Paired comparisons are consistent if the inconsistency rate is less than or equal to 0.1. Otherwise, the decision maker must revise paired comparisons.

4. Results

Result indicated that among the assessed standards, management and empowerment of human resources (weight = 0.465) and requirements and facilities (weight = 0.139) gained the highest and lowest importance respectively. The calculated inconsistency rate is equal to 0.04; accordingly the consistency of criteria with the study aim is acceptable (Table 2).

Table 2. Ranking of the Criteria Using AHP Method.

Criterion	Weight	Ranking
Management and organizing	0.239	2
Management and empowerment of human resources	0.465	1
Safety, quality improvement, and data accumulation	0.157	3
Requirement and facilities	0.139	4

In total ranking, hospitals MS (weight = 0.316), MD (weight = 0.259), AT (weight = 0.14), TS (weight = 0.108), MO (weight = 0.95), and LH (weight = 0.082) acquired the first to sixth ranking respectively.

Considering management and organizing criterion, hospital MS (weight = 0.332) obtained the first and hospital AT (weight =0.07) acquired the last ranking.

Hospital LH (weight = 0.385) obtained the first ranking (weight = 0.437) considering management and empowerment of human resources, safety, quality improvement, and data accumulation criteria.

Considering requirement and facility criterion, hospitals TS (weight = 0.407) and AT (weight = 0.03) obtained the first and last ranking respectively.

Weight and ranking of the other hospitals are represented in Table 3. Performance sensitivity analysis is demonstrated in Figure 2.

Table 3. Ranking of Radiology Departments of Selected Tehran City Hospitals Using ‏AHP.

Hospital Name	Management and Organizing	Management and Empowerment of Human Resources	Safety, Quality Improvement, and Data Accumulation	Requirement and Facilities	Total Weight	Final Ranking
MO	0.100	0.184	0.074	0.159	0.095	‎5‎
MD	0.100	0.032	0.074	0.062	0.259	‎2‎
MS	0.332	0.04	0.032	0.095	0.316	‎1‎
TS	0.199	0.262	0.074	0.407	0.108	‎4‎
AT	0.07	0.97	0.309	0.03	014	‎3‎
LH	0.199	0.385	0.437	0.247	0.082	‎6‎
Inconsistency rate	0.04	0.05	0.02	0.02

Figure 2. Performance sensitivity analysis for priority of radiology departments in selected hospitals.

5. Discussion

Nowadays, health care managers are forced to change their attitude and use new mathematical methods in addition to scientific management for planning (24). In this study, for the first time in the ranking of radiology departments of hospitals, patient and management oriented standards were weighted and compared to each other using comparison matrix, and their ranking was calculated using AHP method. This model enables service receivers to judge the hospitals well and choose the best one to meet their needs (20).

Results showed human resource management and capacity building standards were of more importance compared to other ones. In practice, human resource capacity building can result in organizational effectiveness by facilitating their participation in organizational decision making and work-related affairs. In fact, participation of the personnel in organizational and work-related affairs leads to their individual success in doing their jobs and creates a supportive environment to help organizational performance on a higher level (25). Mosadeghrad (26) states that the quality of health care can be improved by supportive leadership, rational planning, education, effective management of resources, employees, processes, and collaboration of providers of this service. If policy makers and managers intend to improve the quality of health care services, they should apply techniques and tools to operate this quality management structure (26).

The results showed that MS hospital has the best overall performance and the highest quality of service, whereas it was also the best in terms of management and organizing criteria. This could be due to the higher interest of MS hospital in informing the public and personnel about the mission and strategic plan of the organization, and putting educated people in key positions in the radiology department. As Hoe (27) recommends, radiology department managers-who seek to improve quality of their services should clarify their mission and declare it to the employees. Then, they should plan on this basis and implement it in order to ensure quality of services in accordance to regulatory plans such as JCI.

The findings suggested that standards on human resource management and capacity building criteria were met in LH hospital as compared to other hospitals, of which MD hospital had the weakest performance. Human resource planning, supply and distribution were very good and senior managers determined the required characteristics of the employees, necessary feasibility studies were conducted, and the personnel were properly trained at their entry stage and periodically. Documentation of the evidence was fully computer-based and human errors in recording were minimal. Before interventional measures were taken, patients and their attendants were informed and their consent letter was received, while in MD hospital, preliminary personnel training and maintenance of the radiology department were ignored. Keshavarz et al. (28) revealed that standards of patient and their family’s rights, patient health care and training needed to improve in those hospitals, which was consistent with findings of this paper.

Effective resource management can have a considerable effect in customer satisfaction. Satisfied and bound staff will have a better performance in delivering services, which will be followed by better results and higher patient satisfaction (29, 30). In terms of compliance with standards of safety, quality improvement, and data collection, LH hospital was the best, in which all safety codes were met in its radiology department and quality improvement and internal auditing were implemented and reported thoroughly in given intervals. Low rank of MK hospital in connection with these standards was also a result of lack of a quality improvement plan and internal auditing documents in this department. Aseweh Abor et al. (31) showed that hospitals with quality management systems are more successful in implementation of monitoring, leadership and management standards.

The findings showed that standards on hospital requirements and utilities were fully implemented in TS hospital, which received the highest rank in terms of access to emergency and other utilities for the personnel to take safe actions. The lowest rank was recorded in MD Hospital, where the radiology department was completely partitioned from other departments, and necessary emergency and sanitary utilities were not available. Sohrabi et al. (32) suggested that only 26% of hospital had thyroid and gonad shields, lead glasses and apron, which is very disappointing since these protective clothing should be used to protect children and teenagers against the radiation. Focus on protection by the authorities, availability of protective devices, continuous monitoring of concerned authorities on usage of protective devices, and observing sanitary principles could play a major role in reducing the absorbed dose (33).

Mosadeghrad (26) states that productivity decreases when low quality materials are used. Old equipment increase the run time and the results might not be reliable. In a study in India, four hospitals were compared using AHP method. Servqual criteria were used to evaluate the quality of services in the hospitals, which are different from the criteria used in our research. Servqual model is not comprehensive enough and unlike JCI standards, cannot evaluate the quality of hospital services accurately because it is based on interpretation of patients’ perception of quality, which is very difficult to interpret (34). Another study showed that side effects play a critical role in hospital care quality evaluation. Girotti et al. concluded that intensity and the number of effects should be taken into consideration in hospital ranking (35). According to another study, ranking of hospitals is based on an unreliable ranking (36).

In this study, absolute numbers 1 to 9 were used instead of fuzzy numbers. Crisp sets are actually the same ordinary sets that were introduced at the beginning of the sets classical theory. Using fuzzy numbers rather than absolute numbers is the limitation of this study.

The present study used AHP technique to rank radiology departments of hospitals, which can be encouraging to promote the evaluation systems and consequently promote performance of health care systems. The present model can fill the existing gap in hospital accreditation systems related to scoring and comparing hospitals with regard to degrees of importance and value of each standard, consider a combination of qualitative and quantitative criteria, and take very important steps to make health and treatment evaluation systems more efficient and effective. Therefore, the AHP method can be used as an acceptable method to evaluate the quality of services and rank radiology departments around the world.