The next step in learning from sentinel events in healthcare

Abstract

Objective

The aim of this study was to evaluate the current status of handling and learning from sentinel events (SEs) in Dutch academic hospitals and to develop a basis for the first steps in a joint and transparent approach to improve learning from SEs.

Design

Survey by the Netherlands Federation of University Medical Centres (NFU) as part of the project ‘Quality-based Governance’.

Participants and setting

All eight Dutch University Medical Centres (UMCs).

Results

Three methods are used to identify the root cause of SEs: the Systematic Incident Reconstruction and Evaluation, Prevention and Recovery Information System for Monitoring and Analysis or TRIPOD method. Experts with different backgrounds are involved in the analysis of SEs. UMCs have different policies regarding the selection of recommendations for implementation. Some UMCs implement all recommendations formulated by the analysis team and in some UMCs the head of the involved department selects recommendations for implementation. No predetermined criteria have been established for this selection. Most UMCs confirm that similar SEs reoccur, which might be due to the quality of the analysis of the SEs or the quality of the recommendations.

Conclusion

There is a large variety in handling SEs in Dutch academic hospitals and standards for the selection of recommendations are lacking. A next step to decrease the number of (similar) SEs lies in a joint and transparent approach to objectively assess recommendations and further define strategies for successful implementation. Selecting high-quality recommendations for implementation has the potential to lead to a decrease in the number of (similar) SEs and increase in the quality and safety of Dutch healthcare.

Introduction

In the Netherlands, all healthcare organisations are required by law to report sentinel events (SEs) to the Dutch Healthcare Inspectorate (DHI) within 3 days after detection.¹ An SE is defined as an unintended or unexpected event, related to the quality of care, which caused death or serious harm to a patient.² The DHI demands organisations to analyse the SEs through incident analysis to find the root cause or causes and develop recommendations for interventions that prevent or reduce reoccurrence of the SE. It is noteworthy, that despite these efforts similar SEs still reoccur. For example, between 2014 and 2016, 60 cases of wrong-site surgery were reported to the DHI.³

After the introduction of an incident reporting system as part of an obligatory safety management system in all hospitals in the Netherlands, the potentially avoidable mortality—which is measured every 4 years—decreased from 5.5% in 2008 to 2.6% in 2011/2012.⁴ However, in 2017 it became clear that the potentially avoidable mortality did not further decrease. This was one of the main reasons for the Netherlands Institute for Health Services Research to advise the development of a joint approach to improve learning from SEs.⁵ To accomplish this, transparency regarding dealing with SEs between healthcare organisations is needed. This will, in their view, enhance the learning process and thus lead to a decrease in preventable death in Dutch healthcare.

The process of learning from SEs has different stages: reporting SEs, analysing SEs, formulating recommendations, implementing recommendations and evaluating the effect of recommendations (figure 1). Multiplying the quality of these different stages results in the quality of the learning process.⁶ If one of the stages has a low score, the entire process is negatively affected. Due to the directive of the DHI to hospitals to report all potential SEs, first stage of the learning process, the number increased from 773 in 2013 to 1306 in 2016.⁷ The increase in the number of reported SEs does not necessarily mean healthcare became less safe, since this increase is probably due to the fact that SEs are better recognised and thus reported more frequently. The results of the incident analysis done by the hospitals are handed over to the DHI. The DHI assesses the quality of the reports and the following learning process through a questionnaire based on the WHO draft report ‘Concise Incident Analysis’. When received by the DHI, each report gets a score between 0% and 100%. In a period of 2 years, this led to an increase in average score from 64% in 2013 to 78% in 2015.¹ Thus analysing SEs, the second stage of the learning process, seems to be performed adequately. However, there are some uncertainties. As stated above, the increase in score does not necessarily mean an increase in the quality of the reports as the hospitals could have become better at writing the reports in line with the guidelines given by the DHI. Therefore, the methods and the way these are applied should be assessed. To be able to do so, first of all we need to know how hospitals handle the analysis of SEs. Currently, all hospitals report and analyse SEs individually. It is not known what methods are being used to analyse the SEs. Neither is clear who is involved in analysing the SEs or how hospitals deal with the recommendations following incident analysis. Whether they recognise the statement of reoccurring SEs and what they consider to be the cause of this reoccurrence.

Figure 1.

The learning process.⁷ SE, sentinel event.

The aim of this study was to evaluate the current status of handling and learning from SEs in Dutch academic hospitals and to develop a basis for the first step in a joint and transparent national approach to improve learning from SEs.

Methods

To elucidate the current status of handling SEs by Dutch academic hospitals or University Medical Centres (UMCs), a survey was initiated by the Netherlands Federation of University Medical Centres (NFU) as part of the project ‘Quality-based Governance’. The NFU is a federation that focuses on all Dutch academic hospitals, through which we could collect results from within a set time limit. There are currently eight UMCs in the Netherlands, which serve as referral centres for the different regions of the country and also try to set an example for their region with regard to the non-medical side of healthcare, including guarding the quality of healthcare. (The eight UMCs are the Amsterdam University Medical Centres (UMC)—location Academic Medical Centre and location VU University Medical Centre (VUMC) in Amsterdam, the Erasmus Medical Centre (Erasmus MC) in Rotterdam, the Leiden University Medical Centre (LUMC) in Leiden, the Maastricht University Medical Centre in Maastricht, the Radboud University Medical Centre (Radboud UMC) in Nijmegen, the University Medical Centre Groningen in Groningen and the University Medical Centre Utrecht in Utrecht). All are accredited, for example, by the Joint Commission International. Assessing the current status of handling SEs in all academic hospitals was therefore considered a first step in improving learning from SEs nationally.

The survey was developed in cooperation with the NFU, the Chief Medical Officer (CMO) of the Amsterdam UMC—location VUMC and two experts specialised in dealing with human factors during analysing SEs (online supplementary file 1). The main focus of the questionnaire was to collect information about the second, third and fourth stage of the learning process—analysing SEs, formulating recommendations and implementing recommendations—using the following questions: (a) the method used to analyse the SEs; (b) the disciplines represented within the investigation teams responsible for analysis of SEs; (c) how is dealt with the recommendations made by the investigation team; (d) whether or not there is reoccurrence of similar SEs and if so; (e) what could be the cause of this reoccurrence. The CMO and member of the Executive Board in each of the UMCs with the portfolio quality and safety, were asked by email to complete this survey in November 2017 and a reminder was send in December 2017. In January 2018, contact by telephone was sought, as a last reminder.

The survey was subject to a qualitative analysis. Basic descriptive statistics using IBM SPSS Statistics for Windows (V.24.0, IBM, Armonk, New York, USA) were used to create an overview of the current practice in the UMCs. The results of the survey were processed anonymously.

Patient and public involvement

Patients were not involved in this study.

Results

All UMCs responded. The results were collected within 3 months after sending the survey and the following data could be extracted.

Analysing SEs

After an SE, every UMC forms an ad hoc investigation team to analyse the SE. Three methods are used to find the root cause or causes. These are the SIRE, PRISMA or TRIPOD method (table 1).⁸;

SIRE	Systematic Incident Reconstruction and Evaluation (Dutch variant of the root cause analysis from the USA); processes are considered as a whole, each part influences the outcome.	Focuses on the system. Freedom of speech is important. No blaming and shaming.
PRISMA	Prevention and Recovery Information System for Monitoring and Analysis; creates a fault tree and classifies causes in order to develop optimal recommendations.	Focuses on the system. Useful at less extensive incidents.
TRIPOD	Combines risk factors, preventive and corrective measures.	Identifies latent errors and management decisions which contributed to the errors.

Table 1 In six UMCs, this investigation team consists of medical doctors. In four UMCs, at least one of the team members is trained in the investigation method used in that academic hospital. Five UMCs include nursing staff in their teams and in five UMCs quality consultants or medical advisors are included in the team to analyse the SE. Two UMCs have a lawyer, two a pharmacist, one a business administrator and one a medical psychologist in the analysis team. Two UMCs invite a technician when necessary, for example, when medical equipment has failed. One UMC uses the knowledge and experience of external experts when deemed necessary.

Table 1.

Method used to analyse sentinel events (SEs) per Dutch University Medical Centre (UMC)

	SIRE	PRISMA	TRIPOD
UMC 1		♦
UMC 2	♦
UMC 3		♦
UMC 4			♦
UMC 5			♦
UMC 6	♦
UMC 7			♦
UMC 8		♦

The results of the survey were processed anonymously.

PRISMA, Prevention and Recovery Information System for Monitoring and Analysis; SIRE, Systematic Incident Reconstruction and Evaluation.

Formulating and implementing recommendations

In four UMCs, all the formulated recommendations following incident analysis are implemented. In one UMC, a selection of the recommendations as formulated by the investigation team are implemented (table 2). No predetermined criteria have been formulated for the selection of these recommendations. In five UMCs, the head of the involved department selects the recommendations. This selection is based on estimated feasibility (in three of the eight UMCs), urgency (two of eight), potential risk (two of eight), impact, chance of reoccurrence, practical execution and implementation effort (one of eight). In six UMCs, the CMO or Executive Board is in charge to verify if the recommendations are implemented. In three UMCs, the quality and safety department is involved in monitoring the implementation of recommendations. In some cases, recommendations are important for other departments in the academic hospital. All the UMCs discuss if this is applicable. If it is applicable, recommendations will be shared and implemented throughout the academic hospital according to six of the eight UMCs.

Table 2.

Implementation of formulated recommendations following incident analysis per Dutch University Medical Centre (UMC)

	All	Selection	Unclear
UMC 1			♦
UMC 2			♦
UMC 3			♦
UMC 4		♦
UMC 5	♦
UMC 6	♦
UMC 7	♦
UMC 8	♦

The results of the survey were processed anonymously.

Reoccurrence of SEs and possible causes

Five UMCs confirm that similar SEs reoccur (table 3). Aspects such as culture and communication are mentioned as possible causes. Two UMCs believe it could be due to the quality of the analysis of the SEs. Either the scope of the investigation is too narrow or the analysis is not thoroughly enough. According to three UMCs, the cause could also be the quality of the recommendations. They can be, for example, unclear or not suitable for the day-to-day clinical practice. Also, less attention and time than deemed necessary is spent on the implementation of the formulated recommendations.

Table 3.

Reoccurrence of sentinel events (SEs) per Dutch University Medical Centre (UMC)

	Yes	No	Unclear
UMC 1	♦
UMC 2			♦
UMC 3	♦
UMC 4	♦
UMC 5		♦
UMC 6	♦
UMC 7		♦
UMC 8	♦

The results of the survey were processed anonymously.

Discussion

We evaluated the current status of handling and learning from SEs in Dutch academic hospitals to develop a basis for the first steps in a joint and transparent national approach to improve learning from SEs in the near future. Our results show a large variety in handling SEs. Different methods are used and experts with different backgrounds are involved in analysing the SEs. External experts are rarely involved in the analysis process and no predetermined criteria have been established for selecting recommendations for implementation.

Interestingly, five out of eight UMCs confirm the reoccurrence of similar SEs. As stated before, in the Netherlands, the DHI obliges every healthcare organisation to analyse the SEs and develop recommendations for interventions that prevent or reduce reoccurrence of the SE. Because similar SEs seem to reoccur, it is necessary to look into the different stages of the learning process as presented in figure 1. If one of these stages fails, the entire process is negatively affected.

The first stage in the learning process is reporting SEs. The number of SEs reported to the DHI increased from 773 in 2013 to 1306 in 2016. The DHI states two important explanations for this increase in numbers. First and most importantly, the recent directive to report all serious events prior to the analysis—even when it is uncertain it is an SE. And second, the steep learning curve in the recognition and handling of SEs over the past years.⁷

The second stage of the learning process is the analysis phase. Our results show that the UMCs use three methods to find the root causes: the SIRE, PRISMA and TRIPOD method. It may be that the appropriate method is not executed correctly or an inappropriate method is used to analyse the SEs. It is known that specific methods fit specific types of SEs better than others. Thus, it does not seem rational to use one method for every type of SE.⁹ Some methods are better suited for SEs with a technical cause and others for human factors causing an SE.

The third stage in the learning process is the formulation of the recommendations. In the survey, three UMCs indicate that the cause of recurrent SEs seems to lie in the quality of the recommendations. This is supported by a recent publication where the quality of 1137 recommendations following incident analysis was assessed. Only 8% of the recommendations were classified as strong, the remaining 92% were medium or weak.¹⁰ Another finding is that selecting and prioritising recommendations is based on subjective criteria like estimated feasibility and implementation effort. As Testik et al stated, the selection of recommendations is based on the decision makers’ experience and/or subjective preferences.¹¹ When the quality of recommendations can be assessed objectively, time and money—both scarce in healthcare—can be used more efficient. In our view, this is a stage within the learning process with great potential, where improvement is expected to cause important results in the safety of patient care.

Also, in this survey, multiple UMCs suggested that less attention and time than necessary is spent on the fourth stage—the implementation of the formulated recommendations. Successful implementation is, as well as the other stages, critical in the learning process. Quality improvement simply cannot be achieved when recommendations are not being implemented. In the future, we should further define strategies for successful implementation to accomplish quality improvement.

Apart from the issues stated above, there are other issues to address. The human factor plays an essential and crucial role when it comes to delivering safe care in hospitals.¹² Human factors have been identified as a field with the potential to significantly increase the quality and safety of healthcare.^13–15 However, the integration of human factors in healthcare is stagnating and the full potential of human factors science in achieving safe care is not used.^{16 17} This is supported by the results of this study, since the UMCs have indicated that the members of the local investigation team are not specifically trained in dealing with human factors nor is a human factors specialist part of the team. This contradicts with recommendations from the Institute of Medicine on applying human factors for safe care, as well as examples from other industries.¹⁸ The addition of experts, with a focus on human factors, to the team can increase the quality of the investigation. Examining problems from a different point of view might also help in the formation of suitable recommendations.

A limitation of our study is the fact this is a single nation, single hospital type and small sample size study. In the Netherlands, all healthcare organisations are required by law to report and analyse SEs and develop recommendations for interventions that prevent or reduce reoccurrence of the SE following a strict format.¹ We therefore may assume that processes in all hospitals, both academic and non-academic, should roughly be similar. However, in academic hospitals, the number of SEs are higher due to higher volume and more complex cases. Also, the resources available for incident analysis might be different in academic hospitals compared with non-academic hospitals. This might cause variation in the processes when comparing academic and non-academic processes. Notably, our results already show a large variety in handling SEs within the academic hospitals, in which comparable results could have been presumed, and this variation will probably be even greater in non-academic hospitals or countries in which healthcare is otherwise organised. Our results reveal there is still a lot to be done to improve the learning process in handling SEs and to accomplish a joint approach.

Conclusion and recommendations

There is a large variety in handling SEs in Dutch academic hospitals and standards for the selection of recommendations are lacking. A next step to decrease the number of (similar) SEs lies in a joint and transparent approach to objectively assess recommendations and further define strategies for successful implementation. Selecting high-quality recommendations for implementation has the potential to lead to a decrease in the number of (similar) SEs and increase the quality and safety of Dutch healthcare.