Developing a Model for Accreditation of Iranian Teaching Hospitals: A Qualitative Study

Abstract

Background

Among different factors, accreditation is being widely used across the world to improve quality and safety in hospitals. Therefore, the purpose of the present study was to develop an accreditation model for teaching hospitals in Iran.

Methods

This qualitative study was conducted in four phases from January, 2017 to March, 2018. To this end; firstly, existing accreditation models were extracted and reviewed comparatively. Within the second stage, dimensions and components of the accreditation model were extracted through semi-structured interviews. In the third stage, a new instruction was developed via integrating the findings from the first and the second stages. Finally, the model was validated in two phases of Delphi method and a specialized forum in the fourth step. Qualitative findings were then analyzed using content analysis method.

Results

Models of Joint Commission International (JCI) and Word Federation for Medical Education (WFME) in other 6 countries were reviewed and compared with the current Iranian model. Extracted dimensions discovered to complement the present model included learner assessment, continuous reviews and revisions, and educational productivity. The final model was also developed with 12 dimensions and 94 standards. Content validity ratio (CVR) and content validity index (CVI) were also estimated to be 0.40 and 0.80, respectively. As well, the second round of Delphi method could increase the number of model standards to 97. Moreover, Cohen's kappa coefficient was calculated to be at least 0.71.

Conclusion

This study led to the development of a comprehensive model for scientific accreditation of teaching hospitals through reviewing documentation, combining and comparing global approaches, as well as integrating them with the views of domestic experts.

Introduction

Hospitals play an important role in preventing, treating, and rehabilitating patients; and also the bulk of healthcare resources are spent by them (1–3). In this regard, quality control of health services is considered as the first step in providing effective services for better responsiveness (4). The application of standards is similarly one of the strategies to achieve appropriate levels of quality (5). Additionally, standards are regarded as expectations that are designed to ensure quality of service (6). Within health systems, strengthening an evaluation system is known as one of the most effective tools used to attain a responsive and effective system (7). Accreditation is currently one of the most widely used systems for evaluating health systems. The given approach is exploited in most countries due to its positive impacts on healthcare indicators (8–10). Accordingly, accreditation of hospitals can bring about operational effectiveness (8), professional development (11), reinforced inter- and intraorganizational relationships (12), development of quality and safety-oriented culture (13), increase in compliance with safety standards (14), improvement in outcomes of patients (15) and their satisfaction (16) as well as an enhanced public image of hospitals (17).

In this regard, Joint Commission on Accreditation of Healthcare Organizations (JCAHO) established Joint Commission International (JCI) to respond to the growth in global demands for standardized assessment in healthcare organizations (18). For this purpose, the JCI investigated 500 international healthcare organizations in 2013. In this case, there have been numerous studies evaluating the impacts of external accreditation systems on hospital performance and patient outcomes (8, 19–21). The concept of accreditation also refers to the systematic assessment of hospitals using certain and explicit indicators, and it is considered as a process implemented by an independent organization based on codified standards in order to evaluate units of an organization and decide on granting executive competence to that organization (22).

For the first time, Iranian hospitals were evaluated with a limited number of structural standards in 1962. Then, structural and procedural standards were developed in 1997; and finally an accreditation system was established in 2012. It should be noted that the tasks of policy-making, planning, and directing accreditation in Iran are assumed as the responsibilities of the Office for Accreditation of Healthcare Institutions in the Ministry of Health, Treatment, and Medical Education (MOHTME) (23). In this respect; medical accreditation standards for hospitals were publicized in Iran in March 2011 (24). Over time, accreditation standards of Iranian hospitals have also seen a number of revisions to their comprehensiveness (25). However, there has always been a challenge for teaching hospitals because of their wide variety of activities and measures. A teaching hospital is one that has educational and research responsibilities for training doctors, rescuers, medical and paramedical students, and also providing advancement in medical science along with treating patients (26).

Teaching hospitals are also the main providers of care services in the public sector of Iran; therefore, operation of these centers has a direct and significant impact on health system (27). Teaching hospitals now account for about 50% of hospital beds and 68% of academic beds (28). Due to the diversity of missions as well as insufficient transparency of relevant rules and regulations, such hospitals are exposed to numerous problems in their roles, which need to be addressed and resolved. In this regard, one of the main issues is the accreditation of these hospitals as teaching and research institutions providing health services (29). Multiple measures have been correspondingly taken in the domain of accreditation of teaching hospitals in Iran following international programs. It should be noted that criteria and measures regarding the accreditation of teaching hospitals were developed by the MOHTME and announced for the first time in 2015 after reviewing international documents as well as holding scientific and specialized meetings (30). At the same time; following evaluation of hospitals by universities in Iran and providing feedback on the results, hospital accreditation guidelines were reviewed and their third generation was developed and presented in 2016 (31).

It is noteworthy that old standards in the domain of education have a structure-oriented nature, and there is still no fundamental standards regarding outcomes and educational processes in the world (32). However, examples have emerged in the last decade on accreditation standards of teaching hospitals and medical education. For example, Huang et al. developed accreditation standards for teaching hospitals in Taiwan (33) and the Word Federation for Medical Education (WFME) codified standards for medical education (34). However, none of previous studies on accreditation in Iran had pointed to educational accreditation (18, 35–38). Some of these research studies had been solely conducted for the purpose of adapting the standards employed in Iran to international standards, and no specific standards had been developed for teaching hospitals (18). Some other investigations had also provided models for accreditation of health system in its broad terms in academic centers (38), which could not be applicable to assess teaching hospitals. Based on the reported results of studies conducted in Iran and other countries, tangible points could be addressed in the domain of accreditation of teaching hospitals in Iran. Firstly, Iran had no specific program for accreditation of these hospitals before 2012. Secondly, instructions used in teaching hospitals by Iran after this period had been taken from models in other countries, and given the reviews, they were still not well suited to the status of Iranian hospitals. Finally; design and development of a localized model, considering current laws and regulations, expert opinions, as well as those of executives as the most related ones to accreditation program, could lead to the development of a comprehensive model and provide conditions for the effectiveness of accreditation to improve the status of medical education in Iran. Therefore, the present study was conducted to respond to this issue and to design an accreditation model for Iranian teaching hospitals.

Materials and Methods

The current qualitative study was conducted from January 1st, 2017 to March 6th, 2018. To this end, four stages were taken to develop the accreditation model. In the first stage, existing accreditation models of teaching hospitals and medical education were extracted through a comprehensive review. For this purpose, the search was carried out on websites of relevant international institutions, databases and websites of health ministries, as well as the ministry of higher education in selected countries. The criterion for selecting the given countries and models of accreditation was accessibility to information. In the case of countries, pioneers in the domain of medical education and its accreditation as well as regional countries similar to Iran were selected. The models of relevant international institutions were also extracted to compare them with the current Iranian model. In this stage, the dimensions and components of the selected models were extracted and compared, and finally the shortcomings of the current Iranian model were delineated.

In the second stage, semi-structured interviews (with open-ended questions about the drawbacks of the current model and suggestions for complementing its dimensions) were used to extract the components and the dimensions required for the accreditation of Iranian hospitals. Given the nature of the required data as well as the objectives of the study, purposeful sampling method was employed. Moreover, there were attempts to use key informants for interviews and finally 19 interviews with a mean duration of 45 minutes were fulfilled until data saturation. Manifest content analysis method was also utilized to analyze qualitative data. In this method, semantic units were initially extracted and then classified and merged to form categories and subcategories. Finally, the main themes were specified by examining overlapping and semantic relationships of the categories. The criteria provided by Guba and Lincoln reporting that there were four criteria for increasing reliability and credibility in qualitative studies including credibility, confirmability, dependability, and transformability were similarly used to increase rigor in research (39). In this regard, the results were given to the interviewees during the interviews, after their completion, and following their analysis; feedback was received, and then they were corrected if there were contradictions. All the research team participated in its implementation and analysis. Three faculty members outside the research team also contributed to this study. The coding procedure was performed separately by two researchers. In all cases, there were over 90% agreements in the comments. Moreover, direct quotations were used and enough time was allocated to all stages of the study. Participation in interviews was fulfilled with prior coordination, agreement on time and place, as well as rights to interrupt the interviews by the interviewees. Data were then analyzed using MAXQDA-10.

In the third stage, the findings from the first and the second stages were integrated by the research team to develop a new model for the accreditation of Iranian teaching hospitals. In this stage, overlaps and drawbacks of the current model used in Iran and international models in selected countries were extracted. Then, the dimensions and components of the accreditation model were formulated by integrating them with the findings from the analysis of qualitative data obtained from the interviews.

In the final (fourth) stage, the model was validated by experts to address the deficiencies. In the first phase (the first round of Delphi method), a template was developed in the form of a survey by experts in which dimensions and items in the primary accreditation model were scored based on necessity, relevance, clarity, and simplicity of each item by experts. At this phase, content validity ratio (CVR) and content validity index (CVI) were employed for model validation (40). A total number of 30 experts who had experience in authorized specialist work teams or had published research articles in the domain of accreditation, or were involved in accreditation units at the MOHTME or medical universities participated in this phase. During the second round of Delphi method, the results of the first round were shared with the participants who were asked to submit their suggestions to complement the extracted model. A total number of 12 participants also submitted their own comments for completing the developed model. After collecting and analyzing the suggestions by the research team, the required modifications were applied in the developed model. In the final phase of the model validation, a meeting was also held with the presence of 9 experts, during which the proposed model was evaluated by calculating CVI and modified Cohen's kappa coefficient. At this phase, the status of the extracted model was evaluated for applicability, adherence to top-level documents, acceptance by beneficiaries, simplicity, coherence and integrity, as well as comprehensiveness. Additionally, in an overall item, the opinion of participants was assessed about the suitability of the extraction model for teaching hospitals. The approach developed by Polit (2007) was correspondingly used to calculate CVI and modified Cohen's kappa coefficient. According to Polit, Cohen's kappa coefficient ranged from 0.40–0.59 as a fairly good index and it was good and excellent if it was at the range of 0.60–0.74 as good and over 0.75; respectively (41). Inclusion criteria for participants in the specialized forums were similar to the ones defined for participation in Delphi method.

The ethical considerations of this study were reviewed by the Ethics Committee of Iran University of Medical Sciences (code: IR.IUMS.REC1395.9223652205). An informed consent was also obtained from all individuals. Furthermore, they were ensured for the confidentiality of information.

Results

Along with the current model used in Iran, eight models were extracted by searching and considering the defined criteria (accessibility to information, models of relevant international institutions, pioneering of the selected country in the domain of educational accreditation, and similarity of the countries with Iran). The accreditation model of academic medical centers was first presented by the JCI in 2012 and then released in 2017. A model of the WFME in the field of educational accreditation was also selected for evaluation. Similarly; the models from the United States, Canada, Australia, and Japan were extracted as countries of interest in this field, as well as from Turkey and Taiwan as similar regional countries. The results of a comparative study of the models extracted and the model used in Iran in terms of dimensions and core components were presented in Table 1 which suggested no financial support and counseling for students, interactions with society and industry, and internationalization in the model used in Iran. The participants in the interviews on the extracted dimensions included 19 individuals. Most of them were under the age of 40 years (36.8%) and male (62.2%). Almost three-quarters of these individuals (73.7%) were holding Ph.D. degree (clinical and non-clinical). All the participants with Ph.D. degree in non-clinical major were graduates of healthcare management. Most of these participants (36.8%) also had working experience of 10 to 20 years and this value was less than 10 years in 78.8% of the individuals in their current position. Most of the participants agreed on quality and quantity of dimensions and measures regarding the accreditation of teaching hospitals. In some cases, the participants provided recommendations to improve the nature and the content of the instruction associated with educational accreditation.

Table 1.

Comparative study results of selected educational accreditation models

In this context, some participants emphasized the active role of ethical components in research into teaching hospitals. In this respect, one of the participants said that: “Limited attention has been paid to the role of research. Research into teaching hospitals is also commonplace. The dimensions of ethical considerations and thus the material and moral rights of participants must be explicitly monitored and evaluated” (Participant No. 5).

In the field of research; considering strategies to attract funding sources, moving in the direction of obtaining grants, and holding relationships with the industry were presented as suggestions. For example, a participant reiterated that: “There are different conditions to fund universities in advanced countries. Universities, as firms, must be able to meet their costs and be profitable institutions. Some actions have been also developed towards this direction in Iran; the speed and the strength of these movements can be enhanced through inclusion in accreditation programs. Additionally, more points should be given to the centers wherein industry is flourishing and research needs to be conducted through attracting grants so that differences can induce motivation” (Participant No. 14).

Another issue noted by the given participants was the ongoing reviews and revisions of educational programs for students within hospital environment. Accordingly, it was stated that: “Practical training takes place entirely in hospital environment. Therefore, new principles of education should be used in the clinical and theoretical fields. There should be further codified programs and mechanisms for continuous reviews and reconsideration of educational programs and contents. The training methods and contents used should be always reviewed. Existence of educational development units in hospitals or close collaboration of centers and colleges with hospitals can be useful in this respect” (Participant No. 11).

Educational achievement and monitoring were other dimensions mentioned by the interviewees. “The funding system through public resources may not provide sufficient incentives for productivity promotion interventions at universities and teaching hospitals. However, it should be noted that costs can be reduced in most cases, and a certain number of students can be trained effectively at varying costs, so the lowest costs will always be more acceptable if the quality is satisfied. These items should be considered in educational accreditation. In these circumstances, several plans are required to improve productivity” (Participant No. 13).

The findings showed that accreditation guidelines should be more in line with the conditions of hospitals in Iran; educational centers in this country can adapt themselves to the given status while respecting standards and taking them into consideration. This factor also requires the integration of existing guidelines with international standards in the domain of medical education as well as the views of experts and professionals in order to formulate guidelines in an appropriate manner.

Synthesizing the findings from the interviews and combining them with international accreditation models as well as the latest model employed in Iran, the raw model for accreditation of teaching hospitals was extracted with 12 dimensions and 94 standards, including 1- Governance and educational management, 2- Monitoring and evaluating educational system, 3- Evaluation of students, 4- Faculty members, 5-Learners, 6- Management of facilities, spaces, equipment, and financial and human resources of education and research, 7- Training learners in emergency and para-clinical departments, 8-Educational programs and processes, 9-Respecting patient rights in educational processes, 10- Considering teaching hospitals as an area of clinical research, 11- Educational productivity, and 12- Continuous reviews and revisions.

During the first step of the validation model (the first round of Delphi method) in this study, there were 30 participants in the accreditation program, including 10 executives and managers, 10 faculty members, as well as 10 key staff members. Moreover, 60% of the participants were within the age of less than 45 years, 60% of the individuals were male, and 40% of them were women. The proportion of participants with bachelor's and master's degrees was equal (each one was 20%). The rest of the participants were general practitioners and Ph.D. graduates (each one was 23.33%), while the proportion of clinical specialists was the lowest (13.34%). In addition, 43% of these individuals had working experience between 10 and 20 years. Based on the number of survey participants (n=30), the minimum acceptable CVR was 0.33 and the minimum acceptable CVI was equal to 0.79. The results also showed that the minimum and maximum CVR in 94 standards tested were 0.40 and 0.80, respectively; hence, all the items were confirmed in this regard. The minimum and maximum CVI were estimated by 0.80 and 0.95, respectively; so that was at an acceptable level for all items. In the second step of validation (the second round of Delphi method) for the developed model, the results of the first step were presented to the participants who were then requested to provide their suggestions to complement the extracted model. A total number of 12 participants also submitted suggestions for completing the developed model. After collecting and analyzing the proposals by the research team, three items were added to the original model including a standard in the domain of monitoring and evaluating educational system, a standard in the domain of continuous reviews and revisions, and a standard in the domain of educational productivity. Eventually, the number of standards developed for the model was increased to 97.

In the next step, for the final validation of the model, a meeting was held with the participation of nine experts consisting of 5 staff from the MOHTME, 2 faculty members, and 2 staff from the teaching hospitals. The mean age of the participants in this step was estimated to be 43.6 years, 6 (66.66) of them were male and 5 (55.55) individuals were general practitioners or holding Ph.D. degrees. The results of the reviews of the CVI and modified Cohen's kappa agreement coefficient were shown in Table 2.

Table 2.

Comparison of opinions and agreement among experts on the validity of the proposed model

Dimensions	I-CVI	K*	Scores
Applicability	0.90	0.89	Excellent
Adherence to top-level documents	0.90	0.71	Good
Welcoming by beneficiaries	0.80	0.86	Excellent
Simplicity	0.90	0.85	Excellent
Coherence and integrity	0.80	0.81	Excellent
comprehensiveness	0.80	0.89	Excellent
Suitability of model for Iranian teaching hospitals	0.90	0.91	Excellent

Considering the above findings, the developed model had an acceptable status for six factors. In general, participants agreed on the suitability of the extraction model for teaching hospitals in Iran. Based on various validation steps, the final model for the accreditation of educational hospitals was approved with 12 dimensions and 97 standards (Table 3).

Table 3.

Dimensions and number of standards for the accreditation model of Iranian teaching hospitals

Dimensions	Number of standards
Governance and educational management	12
Monitoring and evaluating the educational system	5
Evaluation of students	6
Faculty members	13
Learners	9
Management of facilities, spaces, equipment, financial and human resources of education and research	13
Training learners in emergency and para-clinical departments	6
Educational program and process	11
Respecting the rights of patient in the educational process	8
Considering the teaching hospitals as an area of clinical research	7
Continuous review and revision	3
Educational productivity	4
12 dimensions	97

Discussion

In this study, the accreditation model of Iranian teaching hospitals was developed through four stages in the form of 12 dimensions and 97 standards. The results of the components of educational accreditation models also showed that there were components of governance and leadership, missions and objectives, learners, student evaluation and its mechanisms, and educational environment and resources in all the extracted models. Governance and leadership have been so far considered as the most effective and necessary components in validation systems. In this context, the main custodian of education in a hospital should be identified, and the roles should be formulated in a clear and defined manner. Transparency of responsibilities and roles can also increase monitoring potential and reduce probability of negligence (33, 35, 42). Similarly, evaluation of students and related mechanisms has been among the most significant joint items in all extracted models. In this respect, students are considered as clients and ultimate products of medical education systems. A codified and comprehensive assessment system should be also available to truly measure the capabilities of medical students and enable the possibility of improving the status quo, in addition to realizing the principles such as merits and fairness. The last joint item in the extracted accreditation models was facilities, environments, and resources of education and research. Undoubtedly, the educational environment and resources are of the most effective components in the domain of training human resources. Educational environment can also affect attitudes and motivations of students as well as educational productivity (43).

The subjects of interactions with society and industry, internationalization, and patient rights in were the least frequent ones in educational processes. Today, universities are moving beyond their second generation and directing towards Third Generation Universities (TGUs). One of the characteristics of TGUs is a potential and high-level interaction with society and industry. The TGUs are also moving towards self-sufficiency in financing, and this necessarily involves moving towards interacting with industry and commercializing science (44). On the other hand, attention to society as a fundamental principle in teaching medical science has always been considered. Community-oriented medical education can also help educational systems meet the needs of populations efficiently. One of the goals of integrating education and treatment in 1985 in Iran and its establishment by MOHTME has been providing motivations to drive medical education towards society and its real needs (45).

Internationalization of education was another item with the lowest frequency among the models. Currently, economics of education has become one of the most commonly used terms in the world. Pioneering in the field of education can thus become a benefit to attract students from other countries, which will definitely have an effect on economics of education and other economic areas of countries (46). Considering patient rights in medical science education was the third item with a low frequency among the studied models. However, since the 18th century and along with the development of therapeutic and anatomical methods, discussions on patient rights have gradually become the focus of attention. The need for students to attend patient bedside and their access to patients' personal information require clarity in order to provide confidentiality of patient information. Moreover, conditions must be such that prescribing treatments and their implementation by students are carried out under the supervision of a competent person and with full knowledge of patients. With regard to medical research, patient rights must be met in the field of awareness on research and its objectives as well as outcomes and other necessary information (47, 48). The analysis of interviews resulted in the extraction of three dimensions of accreditation model of teaching hospitals. One of the most prominent items to consider was focus on research issues in teaching hospitals. Funding research projects and likelihood of attracting grants were also the priority of research-related issues. According to the participants; research should be directed towards commercialization and attraction of grants with adherence to ethical and legal considerations (44). The TGUs-related policies in the MOHTME Department of Education in Iran have also been discussed as evolutions in education. Continuous reviews and revisions in educational programs and related processes have been also the main dimensions proposed by interviewees to add to the current educational accreditation model. The rapid and growing trend of science production, especially in the domain of medical sciences, requires continuous consideration of educational programs and contents, as well as clinical training methods and revisions on the basis of the best available evidence (49). Another area proposed by the interviewees to complete the accreditation model was educational productivity. Today, the economics of education is similarly regarded as one of the most globally well-known economic areas. Despite the aspects of income, expenditures and cost management in this branch of economy are of utmost importance (46). At present, costs of education are on a rise considering advances in medical education and specialization of this knowledge (50). Therefore, attention to educational productivity is essential for achieving sustainability in resources and possibility of growth and development.

Moreover, the feedback of findings from the first step of validation by the participants and the demands for suggestions in completing the model led to the addition of three items to monitoring and evaluation domains of the program, continuous reviews and revisions, and educational productivity. Monitoring and evaluation of the program was also added by the item of the system of sharing the results of the evaluation program and the conditions for application in order to improve the status. Benchmarking is also one of the low-cost and effective strategies to use the successful experiences of similar systems (51). The organizational conditions were almost the same in teaching hospitals and medical faculties. Therefore, there is the possibility of transferring experiences that can result in effectiveness of training and declining costs by reducing trial and error. Continuous reviews and revisions were also strengthened by the item of mechanisms used to apply international (evidence-based) scientific and empirical advances in education to revise and enhance educational programs and processes. As noted earlier, implementation of successful experiences and modeling was considered as a low-cost approach to improve the status quo. Finally, an item entitled conditions for transfer of experience and cost control in educational centers was added to the domain of educational productivity. Hospitals and academic centers can thus transfer their own experiences in reducing and managing costs during reasoning and specialized sessions, and can consequently predispose the enhancement of educational productivity through sharing knowledge. In extensive systems such as college systems, the transfer of knowledge and experience as one of the areas of knowledge management could promote efficiency and effectiveness (52). Finally, the triple process of validation resulted in the approval of the accreditation model of Iranian teaching hospitals in 12 dimensions and 97 standards. The current model used in Iran was comprised of 9 dimensions and 81 standards (31) whose quantitative and qualitative development could improve the state of executive conditions and the consequences of accreditation in teaching hospitals using the new model of accreditation.

The onset of each program would face many challenges and shortcomings that should be recast and completed through frequent revisions, forums and discussions, as well as use of international experiences. The experience of educational accreditation in Iranian university hospitals is a new step as well. The model used in this regard was based on patterns from other countries. Despite extensive efforts to nationalize this model and its adaptation to the conditions of hospitals in Iran, there are problems with the implementation of accreditation as well as its consequences. Through reviewing documentation, combining and comparing global approaches, and integrating them with the views of domestic experts; the results of the present study could scientifically predispose a basis for improving the status of educational accreditation, quality of services, and safety in Iranian hospitals.

The present study had limitations in terms of its external validity given the nature of its design (qualitative research). One of other limitations in this study was that accreditation models were selected among a number of countries and international organizations which might not be a representative of entire countries and related organizations across the world. Differences in the instructions of other countries and organizations could also increase the comprehensiveness of the model designed in this study. Moreover, since the introduction of the current model for accreditation of teaching hospitals in Iran, it might have affected the mentality of participants and challenged their creativity in terms of expressing new ideas. So, there were attempts to compensate somewhat for this limitation through conducting the study in four stages and selecting samples from different levels.