A self-assessment model for hospital safety culture maturity

Abstract

Background: Studies on safety culture maturity in health care is very rare, and the existing ones only focus on patients and the use of Manchester Patients Safety Framework (MaPSaF) instrument. The objective of this study is to develop a comprehensive instrument for measuring safety culture maturity in hospitals.

Design and methods: This study used a cross-sectional design with three stages. First, we used secondary data analysis from the Hospital Accreditation Commission. Second, evaluation of primary data obtained from safety climate questionnaire. Third, we did focus group discussions, and in-depth interviews for validation of secondary data and development of DUTA-RS website. We analyzed using Structural Equation Modeling (SEM) test.

Results: DUTA-RS instrument contains 1,118 elements based on the first edition of the Indonesian Hospital Accreditation National Standard. Its safety culture maturity is at the proactive level (58.0%), with the highest accreditation levels of proactive (50.8%) and generative (48.7%). The variables affecting the safety culture maturity are situational and safety behavior variables, with leadership, risk management, and safety compliance as the strongest indicators. The weakest indicators of climate are organizational learning and communication. The mean value of climate for primary and secondary data is in the good category and showed in proactive level.

Conclusions: The DUTA-RS as a website to measure the safety culture maturity in accredited hospitals by taking the advantage of the existing information technology of hospital accreditation committee as the benchmark enables improvement of SCML in hospitals. Further studies are required for the development of DUTA-RS website.

Significance for public health.

Accreditation is an essential element to assess the quality of the hospital in the safety culture. Implementing the accreditation in hospital requires several standards on maturity safety culture guidelines. This research shows that safety culture is important for patients and their families. Besides, we have tools for assessment and evaluation of safety culture in hospitals that can increase the hospital's reputation because patients, families, visitors, and the community feel comfortable and safe in the hospital environment that already has a good safety culture maturity. This model can be used by hospitals as a recent comprehensive instrument to measure the maturity level of hospital safety culture including hospital quality, patient safety, worker safety, and health.

Introduction

Hospital is a health care facility that has the obligation to protect patients, community, environment, and its human resources due to the inherent risks of medical procedures that affect patient safety and may lead to occupational accidents and occupational diseases. The National Patient Safety Committee data presented an increase in reports of patient safety incidents in hospital for the period 2015-2019.¹ Although the number of reports during this period was not as high as in another report in 2013, which presented the highest number of reports, the number of reports submitted during the period is still alarming.². Injuries and illnesses among workers in health services are two times greater than in other industries, with the prevalence of needle-stick injuries that reaches 50%.^3,4

Safety culture is a concept that was first introduced after the Chernobyl nuclear accident in Uni Soviet in 1986. It is a very important concept for improving the safety and health of the employees during work, which will eventually improve the organizational performance. A safety culture model has been introduced to measure the safety culture in an organization, which explains three variables that interact with each other, namely safety climate (perceptions), situational (management systems), and safety behavior. A poor safety culture reduces service quality while good safety culture will prevent accidents.^5,6

Safety culture maturity reflects how a safety culture is developed dynamically by following the process and implementation of the safety management system in the organization.⁷ Westrum in 1988 (Organizational and interorganizational thought. Presentation to World Bank conference on Systems Safety) first divided safety culture maturity into three levels (pathology, bureaucratic and generative) but later added two more levels (reactive and proactive), making a total of five levels.⁸ Since 2000, the safety culture maturity model has been used by offshore, construction, petroleum, and healthcare industries, to gain insights for improvement towards a better safety culture maturity level.^9,10 Studies on safety culture maturity in health services are still scarce, based on the a systematic literature review using five databases for 2009-2019, with the existing studies only focuses on patient safety and uses the Manchester Patient Safety Framework (MaPSaF) instrument.¹¹

In Indonesia, hospitals are required to undergo an accreditation process by an independent accreditation agency, such as Hospital Accreditation Committee (Komite Akreditasi Rumah Sakit, KARS) that has received international recognition from the ISQua External Evaluation Association (IEEA). This requirement for accreditation is the government’s effort to improve health services in Indonesia.

This study has demonstrated that the accreditation received by a hospital influences the frequency of reporting and safety perception in the hospital.¹² Hospitals that have received accreditation will achieve the safety culture maturity within three years if they continue to implement the accreditation standards.¹³ Recently, the instrument that used assessment safety culture maturity was MaPSaF which only measured patient safety. In Indonesia, the existing safety culture for hospital assessment system just only based on perspective patients (¹¹), but in this study the instrument looks based the perspective of patients and workers and also hospital quality.

Therefore, this study aims to develop a model for assessing safety culture maturity, create variable constructs, determine the level of safety culture maturity, and assess the influence of safety climate, situational (safety management systems) variable, and safety behavior on safety culture maturity in hospitals. This model, which is referred to as DUTA-RS (Dewasakan Upaya Tatanan Akreditasi Rumah Sakit) instrument, makes use of the information technology to adapt to the digitalization 4.0 era by making it online so that it is accessible to hospitals that would like to perform selfassessments as an effort to maintain the quality and safety by improving the safety culture maturity level.

Design and methods

Study design and participant

This is a cross-sectional study on primary and secondary data collected during the period of April to December 2020. The primary data were collected from three hospitals selected using the purposive sampling method, i.e. Cipto Mangunkusumo General Hospital, Universitas Indonesia Hospital, and Pertamina Jaya Hospital, while the secondary data were obtained from the Hospital Accreditation Committee. The secondary data consisted of data from hospitals all over Indonesia that were regularly accredited by the committee from 2018 to 2019 (n= 1,291). After the inclusion criteria were applied, 708 hospitals were included in the study, consisting of hospitals in the plenary level (highest level), main level, intermediate level, and primary level. The classification of the safety culture maturity used in this study classified the hospital safety culture maturity into pathologic (level 1), reactive (level 2), bureaucratic (level 3), proactive (level 4), and generative (level 5).¹¹

Data collection

This study was conducted in three stages. First, the secondary hospital accreditation data from the Hospital Accreditation Committee were used as the basis for developing the safety culture maturity instrument.¹⁴ This study used the DUTA-RS (Dewasakan Upaya Tatanan Akreditasi Rumah Sakit) instrument, consisting of 1,118 of 1,346 assessment elements of the national accreditation standard instrument, or SNARS, as compiled in the 1^st edition of SNARS from KARS. The DUTA-RS instrument comprises of 4 (four) variables including safety climate, situational (safety management system), safety behavior, and safety culture maturity, with 15 indicators.^11,14,15

The Hospital Accreditation Commission (KARS) as an independent national accreditation institution in Indonesia has been accredited by the IEEA International Accreditation Board (ISQua External Evaluation Association) and received 3 (three) awards from ISQua including for Organization, Standards, and Surveyor Training Program. Content validity was performed through an expert panel of KARS to determine the elements that would be used in the study. Five maturity levels, as determined by the mean score from the element indicator assessment results, were then assigned as follows: Pathology (0% - <20%), Reactive (20% - <40%), Bureaucratic (40% - <60%), Proactive (60% - <80%), and Generative (80% -100%).¹⁴

The second stage of the study consisted of primary data collection for a post-hoc evaluation of the secondary data validation. Data were collected using the safety climate questionnaire and Focus Group Discussions (FGDs). The safety climate questionnaires were distributed from 50-100 workers per hospital while the focus group discussions (7 participants/hospital) were conducted with the heads of units or installations. We convert the results of the FGDs into the score using the Safety Culture Maturity Level (SCML) method.¹⁶ The scores obtained from the safety climate questionnaire were then classified into 5 categories, namely: very poor (score: 1.00-2.00), poor (score:2.01-3.00), fair (score: 3.01-4.00), good (score: 4.01-5.00), and excellent (score: 5.01-6.00).¹⁶

The third stage was the development of the DUTA-RS website for hospital self-assessment and safety culture maturity level improvement. DUTA-RS website is an instrument for self-assessment of the maturity safety culture which includes hospital quality, patient safety, worker safety, and health based on accreditation SNARS (14). DUTA-RS websites consist of 1,118 items questionnaires that 358 items of safety management, 472 items of situational, 154 items of safety culture, and 134 items of safety culture maturity. It was expected that the strongest and weakest factors regarding safety culture maturity would be identified through this website.

Data analysis

Relevant descriptive statistics were performed for participant characteristics and results were presented in average values for continuous variables while frequencies and percentages for the categorical variables. Construct validity was also analyzed using the average variance extracted (AVE) while the internal consistency was analyzed using the composite reliability (CR). In addition, univariate analysis in the form of descriptive, mean, and safety culture maturity level (SCML) was also performed while the confirmatory factor analysis (CFA) and structural equation modeling (SEM) were used for multivariate analysis. After the results of the analyses were obtained and interpreted, the DUTA-RS website for hospital self-assessment was developed. The website format was selected because it was considered to be effective, efficient, safe, high utility, and easy to remember.¹⁷ This website has a dashboard that shows the level of safety culture maturity, average value of the indicators, the weakest indicators per variable, and suggestions for improvement. Data were analyzed statistically using the SPSS ver. 25 and AMOS ver. 22.

Results

The secondary data that covers a five-month period and 708 accredited hospitals were analyzed and it was revealed that the 708 hospitals were in the proactive level of the safety culture maturity level. Most hospitals were also at the proactive level for the hospital quality patient safety, and worker safety indicators (Table 1).

A confirmatory factor analysis for validity and internal consistency was performed by measuring the construct variables of three exogenous variables and one endogenous variable. The exogenous variables consist of safety climate, situational, and safety behavior variables while the endogenous variable was the safety culture maturity. After the results demonstrated valid and reliable variables, a goodness of fit (GoF) analysis by confirmatory factor analysis (CFA) model was performed, resulting in valid and reliable exogenous and endogenous variables (AVE ≥0.3 and CR ≥0.5).^18,19 The cut-off value of Goodness of Fit was met; thus, the model was deemed to be fit. The situational variable has greatest contribution to safety culture maturity (CR = 0.897) (Table 2).

We analyzed the significance level of the estimated parameters of exogenous and endogenous variables in the model. The results showed that there was a significant effect of situational and safety behavior variables on the safety culture maturity. The standard coefficient value shows the effect of the situational (0.596) and safety behavior (0.521) variables. The path coefficient value also determines the direction of the variables.²⁰ It was identified that the strongest indicators were leadership (loading factor = 0.87), risk management (loading factor = 0.87), and compliance (loading factor = 0.85) while the weakest indicators of the safety climate variable were organizational learning (loading factor = 0.62) and communication (loading factor = 0.65) (Table 3).

In order to validate the results of the secondary data, the primary data were analyzed as a post-hoc evaluation. Six indicators were reflected in the primary data. The safety climate questionnaire consisted of 66 items which were responded by 270 respondents from three hospitals. No significant difference was seen in the mean value of safety climate between the three hospitals (sig >0.05) (Table 4).

The results of univariate secondary and primary data (questionnaire, forum group discussions, and interviews) analyses were compared using the Safety Culture Maturity Level (SCML) and categorization.¹⁶ The mean safety culture maturity in primary and secondary data were both at level 4 (proactive). The five indicators were found to be in good category on average, while the leadership and regulatory indicators were in the excellent category (Table 5).

In the third stage of this study, we developed the DUTA-RS website for hospital self-assessment and safety culture maturity level. DUTA-RS websites can be used by all hospitals in Indonesia to measure the safety management, situational, safety behavior, and maturity safety culture. The respondent will fill in the identity data before conducting the assessment. After that, respondents (hospital) answered the questions in the questionnaire per section (safety management, situational, safety behavior, and maturity safety culture). After that, the results of the assessment about the level of maturity safety culture of the hospital will come out. In this website, we can also find out the weakness, strongest indicators and improvement suggestion to increase higher level in each hospital (Figure 1).

Discussion

The results of this study demonstrated that the safety culture maturity level of the 708 hospitals is proactive, meaning that the hospitals only achieve 60% - <80% scores in the 134 assessment elements of the First Edition of the Hospital Accreditation National Standard in maintaining the quality and safety in the hospital. Hospitals with plenary level are found to be in the proactive and generative levels, meaning that the accreditation survey plays a role in improving the quality and safety in the hospital. These hospitals have gone through 1-2 accreditation surveys in 3-6 years period. It is appropriate that the maturity phase will be established within 3 years after accreditation and that it can take 3-5 years to identify changes in the culture.^13,21

Table 1.

The safety culture maturity based on variables.

Level	Variables
	Safety culture maturity	Quality hospitals	Patients safety	Occupation safety and health
	n (%)	n (%)	n (%)	n (%)
Pathologic	0 (0.0)	0 (0.0)	0 (0.0)	1 (0.1)
Reactive	12 (1.7)	21 (3.0)	14 (2.0)	30 (4.2)
Bureaucratic	89 (12.6)	115 (16.2)	92 (13.0)	116 (16.4)
Proactive	408 (57.6)	313 (44.2)	366 (51.7)	306 (43.2)
Generative	199 (28.1)	259 (36.6)	236 (33.3)	255 (36.0)
Total	708 (100.0)	708(100)	708(100)	708(100)

Table 2.

Secondary data validity and internal consistency and goodness of fit.

Confirmatory factor analysis	Goodness of fit
Variable	AVE	CR	Value	Indicator	Criteria	Value
Exogenous variables
Safety climate	0.476	0.843	Valid and reliable	Chi-square: 1.595	< 2	Fit model
				RMSEA: 0.029	≤ 0.05	Fit model
				GFI: 0.996	> 0.90	Fit model
				AGFI: 0.983	> 0.90	Fit model
Situational	0.746	0.897	Valid and reliable	Chi-square: 0	< 2	Fit model
				RMSEA: 0.000	≤ 0.05	Fit model
				GFI: 1.000	> 0.90	Fit model
				AGFI: 1.000	> 0.90	Fit model
Safety behavior	0.688	0.796	Valid and reliable	Chi-square: 0	< 2	Fit model
				RMSEA: 0.000	≤ 0.05	Fit model
				GFI: 1.000	> 0.90	Fit model
				AGFI: 1.000	> 0.90	Fit model
Endogenous variable
Safety culture maturity	0.345	0.610	Valid and reliable	Chi-square: 0	< 2	Fit model
				RMSEA: 0.000	≤ 0.05	Fit model
				GFI: 1.000	> 0.90	Fit model
				AGFI: 1.000	> 0.90	Fit model

AVE, average variance extract; C, composite reliability; RMSEA, root mean square error of approximation; GFI, goodness of fit index; AGFI, adjusted goodness fit of index.

Theoretically, the hospital safety culture maturity at the proactive level means that organizations prioritize continuous safety improvement and reward staff who care about safety; there is enthusiasm throughout the organization for continuous improvement; hospital is open and honest with patients and staff when a patient safety incident occurs; staff feels safe to report safety incidents, hospitals have a culture of learning from reporting safety incidents; communication and records system is fully audited; organizational and individual training needs are identified; there is multidisciplinary teamwork that is collaborative and open;¹¹ a comprehensive positive safety culture approach is available;²² staff are actively involved in all safety issues and processes;²³ the investigative process is conducted openly and involved patients through the implementation of root cause analysis (RCA).²⁴

Table 3.

Structural equation model with standardized regression weights.

Variable/indicator			Standard coefficient/ estimate	Composite reliability	Sig
Maturity	<—-	Climate	-0.133	-0.474	0.635
Maturity	<—-	Situational	0.596	2.405	0.016*
Maturity	<—-	Behavior	0.521	2.170	0.030*
Collaboration	<—-	Climate	0.71
Communication	<—-	Climate	0.65
Work environment	<—-	Climate	0.72
Training	<—-	Climate	0.78
Reporting	<—-	Climate	0.83
Learning	<—-	Climate	0.62
Regulation	<—-	Situational	0.60
Leadership	<—-	Situational	0.87
Risk management	<—-	Situational	0.87
Compliance	<—-	Behavior	0.85
Participation	<—-	Behavior	0.58
Quality	<—-	Maturity	0.77
Patient welfare	<—-	Maturity	0.74
Worker welfare	<—-	Maturity	0.67

*Significant at the 0.01 level (2-tailed).

Table 4.

ANOVA test for primary data.

No	Variable	Mean			Sig
		RSPJ	RSUI	RSCM
1	Management commitment	4.92	5.15	4.99	0.100
2	Safety communication	4.78	4.84	4.71	0.394
3	Rules and procedures (regulations)	4.49	4.53	4.40	0.188
4	Enabling environment	4.57	4.73	4.61	0.213
5	Personal involvement (participation)	4.73	4.72	4.61	0.268
6	Safety training	4.85	4.85	4.64	0.179
	Safety culture score	4.72	4.80	4.66	0.196

RSPJ, Pertamina Jaya Hospital; RSUI, Universitas Indonesia Hospital; RSCM,Cipto Mangunkusumo General Hospital.

Table 5.

Comparison of primary and secondary data with safety culture maturity level (SCML) and categorization.

No	Indicators	Primary data (questionnaire)			Primary data SCML			Secondary data
		(range 1-6)			(range 1-5)			(range 0-10)
		Mean	Level*	Category	Mean	Level*	Category	Mean	Level*	Category
1.	Management commitment	5.05	5	Excellent	4.33	4	Good	7.86	4	Good
2.	Safety communication	4.79	4	Good	4.00	4	Good	7.68	4	Good
3.	Rules and procedures (regulations)	4.49	4	Good	3.88	3	Fair	9.28	5	Excellent
4.	Enabling environment	4.66	4	Good	4.00	4	Good	7.78	4	Good
5.	Personal involvement (participation)	4.69	4	Good	4.25	4	Good	7.72	4	Good
6.	Safety training	4.79	4	Good	4.00	4	Good	6.79	4	Good
	Means	4.75	4	Good	4.08	4	Good	Good	4	Good

*Level 1 (pathology), level 2 (reactive), level 3 (bureaucratic), level 4 (proactive), level 5 (generative).

The safety culture maturity is demonstrated to affect the indicators of hospital quality, patient safety, and worker safety and health in this study. Quality is the key to health services by prioritizing safety, effectiveness, and patient orientation. Patient safety is important in improving quality and reducing the risk of work accidents. ^25,26 There are three stages of safety culture, namely: i) individual perception (safety climate); ii) the safety management system described quality management, risk management, hospital occupational safety and health (OHS), and emergencies; iii) organizational description in the form of understanding and implementing safety culture through FGDs. This is consistent with the three Cooper model interacting variables.¹⁵

Multivariate analysis showed that the safety climate variable has no effect on the safety culture maturity because the OHS application in hospitals has not been optimum. The weakest indicators were communication and organizational learning. A study in Korea stated that supervision and implementation of an OHS system are needed for the development of a positive safety culture.²⁷ There has been a shift in safety culture from bureaucratic to integrated for organizations that focus on safety.²⁸ This is due to the dynamic and unstable nature of the safety climate.^29-31 Hospital can improve the safety culture maturity by using the DUTA-RS website for self-assessment. The government got suggestion about the benefit of accreditation to improve hospital safety culture maturity level. Government updates regulations related to hospital quality, patient safety, occupational health and safety (OHS), for example conducting a policy evaluation to review Minister of Health Regulations No.66/2016 concerning OHS in hospital, this is useful for adjusting and anticipating current and future hospital conditions thus optimizing the OHS implementation which has an impact on the climate/safety culture and the formation of the maturity phase of the hospital safety culture.²⁷ Government as a controller can monitor and evaluate the implementation of hospital accreditation through the use of DUTA-RS website to optimize the implementation of accreditation standards including OHS in hospital. Government should support the hospital to get highest level accreditation.²⁸

Figure 1.

DUTA-RS website. Front view of the website (A), inputting the respondent’s identity or hospital before self-assessment (B), display of fill the items of questionnaires to be answered by respondents (C), and maturity safety culture category from self-assessment from the DUTA-RS website (D).

DUTA-RS website can be used by hospitals as a recent comprehensive instrument to measure maturity level of hospital safety culture including hospital quality, patient safety, worker safety and health. The hospital image increases because patients, families, visitors and the community feel comfortable and safe in hospital environment that already has a good safety culture maturity.³² Academics can explore sources of knowledge from literature and the latest relevant research related to safety culture by using DUTA-RS instrument/website according to the SNARS edition and applicable laws and regulations. The mass media helps to socialize the importance of hospital service system including service quality, patient safety, worker safety and health. It can be to encourage people to speak up about safety culture and participate in realize of safety culture in a good quality and safely.³² Internal or external team collaboration with service units is needed synergistically because a lack of cooperation and an individualistic culture will lead to repeated mistakes.³³ The conducive work environment and facilities are considered to support the safety culture. The work environment also influences the safety culture including, among others, efforts to remind each other of the hazards related to patient safety.³⁴ Work environment conditions are all conditions that affect workers, which comprise of the physical, legal, and responsibility aspects as well as workload.³⁵ Training conducted based on the identification of needs and problems can reduce accidents in the workplace.³⁶ The training aims to equip workers with the knowledge, skills, attitudes, and behaviors necessary for achieving a safe culture.³⁷ Good and fair incident reporting also supports continuous learning and good reporting produces information that can facilitate an assessment of the aspects to be improved.³⁷ This study showed that the role of communication is important for supporting the success of safety culture programs. Safety communication needs to be improved because it can be influenced the size and structure of the organization.³⁸ Incident learning also needs to be improved to prevent recurring incidents. Organizations that adopt a safety culture tend to choose to learn from mistakes rather than blaming. They also approach mistakes in a systemic manner.³⁹ Situational variables contribute to leadership, risk management, and regulation. Strengthening management commitment is needed to improve safety. The development of a safety system can influence the organizational culture.^7,40 The role of hospital leaders is very important in supporting the success of a safety culture program as the commitment and dedication of the leader in implementing a safety culture will inspire workers.⁴¹ Risk management is the activity of creating a risk register and determining the context, as well as identifying, analyzing, and eliminating hazards and risks, supported by conducting coordination, communication, monitoring and review during the process.⁴² Regulation is needed to achieve this as it plays a role as the reference for implementation. Qualitative results revealed that regulation is essential to prevent unsafe behaviors. The occupational health and safety policies in workplaces reflect responsibility, leadership, and commitment to safety.⁴³ Safety behavior variables contribute to safety compliance and participation. Leaders become the role models for implementing safety compliance based on regulations. According to the qualitative results, compliance audits have been carried out on the implementation of health protocols in hospitals, which is seen as important to ensure compliance to the protocols. Compliance is an individual activity to maintain the safety in the workplace by following standard work procedures.⁴⁴

The result in this study shows that workers have been involved in the safety culture program and that the program involves all workers starting from the management level to the implementers. Participation in safety is defined as the involvement of individuals in improving the environment that supports safety.⁴⁵ This is crucial as unsafe work behaviors. such as improper use of personal protective equipment an incorrect needle recapping behavior, can lead to work accidents.

This study had a few limitations. We did not measure the validity of the assessment elements of each indicator. The number of respondents and informants is limited in each hospital, so it is not evenly distributed. We used the First Edition of the Hospital Accreditation National Standard as the reference for the instrument in this study. This standard is not a climate/safety culture or OHS instruments even though it already has the climate/safety culture and OHS standard components; thus, the interpretation is limited.

Conclusions

The dominant safety culture maturity level among hospitals in Indonesia is proactive with situational and safety behavior variables as the strongest variables and leadership, risk management, and compliance as the strongest indicators. The weakest indicators of the safety climate are organizational learning and communication, which require attention because this affects the maturity of hospital safety culture significantly. The qualitative results also show that the role of the leader is very important in supporting the successful implementation of hospital safety culture. It is necessary to collaborate with academics, regional sectors, community hospitals, government (central and local), and the mass media for using the DUTA-RS instrument to improve hospital safety culture maturity level. Further studies are required to develop the DUTARS instrument/website.

Funding Statement

Funding: This research did not receive any specific grant from funding agencies.