Classification Strategies for Non-routine Events Occurring in High-Risk Patient Care Settings: a Scoping Review

Abstract

Introduction:

Non-routine events (NRE) are atypical or unusual occurrences in a pre-defined process. Although some NREs in high-risk clinical settings have no adverse effects on patient care, others have the potential to cause serious patient harm. A unified strategy for identifying and describing NREs in these domains will facilitate comparison of results between studies.

Methods:

We conducted a literature search in PubMed, CINAHL, and EMBASE to identify studies related to NREs in high-risk domains and evaluated the methods used for event observation and description. We applied The Joint Commission on Accreditation of Healthcare Organization (JCAHO) taxonomy (cause, impact, domain, type, prevention and mitigation) to the descriptions of NREs from the literature.

Results:

Twenty-five articles that met inclusion criteria were selected for review. Real-time documentation of NREs was more common than retrospective video review. Thirteen studies used domain experts as observers and seven studies validated observations with interrater reliability. Using the JCAHO taxonomy, “cause” was the most common classification method applied, followed by “impact,” “type,” “domain,” and “prevention and mitigation.”

Conclusions:

NREs are frequent in high-risk medical settings. Strengths identified in several studies included the use of multiple observers with domain expertise and validation of the approach of event ascertainment using interrater reliability. By applying the JCAHO taxonomy to the current literature, we provide an example of a structured approach that can be used for future analyses of NREs.

INTRODUCTION

A non-routine event (NRE) is any departure from what is expected within a system. In a clinical setting, NREs have been defined as an atypical or unusual occurrence that accounts for any disruption in care.^1–5 Before its wider use in medicine, the aviation industry used NRE analysis to identify how teams can address these events and improve performance.⁶ After adoption in medicine in the early 2000s, NRE analysis has been successfully used to perform similar analyses in a range of clinical settings. Unlike adverse event analysis, NRE analysis is not limited to events leading to poor outcomes, which increases the number of observations and potentially reduces bias.^1–3,7 This open-ended approach captures events that are workflow deviations from a pre-defined best practice (e.g., esophageal intubation), unusual events that cause no harm (e.g., provider tripping over wires), or events unrelated to care that create unsafe situations (e.g., power outage).^1,3,5 Most of these studies have found that errors directly causing harm are uncommon, but events that are latent safety threats are frequent and could be precursors to adverse events.^1,5,8–10 Proper identification and analysis of NREs that represent latent safety threats may facilitate the development of approaches for preventing or mitigating associated harm.

Most recent NRE analyses have been performed in high-risk domains where complex and time-dependent tasks are performed, including the emergency department (ED), operating room (OR), and intensive care unit (ICU).^{3–5,8,11–13} Within these settings, several approaches for identifying the significance of NREs have been used, including assessment of the associated time expense and harms, and the underlying causes for event occurrence. No standardized terminology has emerged from these analyses, making it difficult to aggregate the results of studies in these settings. Current terms used for NREs includes “potential adverse events,” “disruptions,” “interruptions,” and “near-misses.”^1–4,13,14 Although these terms correctly identify NREs, the absence of a unified language has limited comparison of the effects of NREs in different studies. Standardization of NRE terminology and the approaches to identifying NREs will address this challenge, especially when assessing the health burden of NREs.

In this scoping review, we evaluated studies of NREs in high-risk healthcare domains and summarized the strengths and weaknesses of the methods used to assess NREs. We applied the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) patient safety event taxonomy to the NREs as an example of a classification method that can be used to clarify language describing these events.¹⁵ We contribute to the patient safety literature by (1) showing the utility of a taxonomy for analyzing NREs, and (2) providing an approach for making comparisons between the findings of existing and future studies.

METHODS

Search Strategy and Selection Criteria

We identified index articles classifying NREs in high-risk patient care settings to initiate the development of search terms.^5,14,16,17 We defined the search using MESH terms from these articles, limiting inclusion to those with non-routine events or similar language in the title or abstract, and to those occurring in high-risk domains such as the ED, OR, or ICU settings. We focused on these healthcare settings because of the complexity and time-sensitivity of associated management and treatment decisions in these domains. To ensure the identification of articles using any term similar to “non-routine,” we also included words such as “atypical” and “unexpected” in the search (see Appendix 1). In December 2018, we searched three databases (PubMed, CINAHL, and EMBASE) for relevant studies, limiting the results to articles published after 2007 and in English. A separate search was performed with key terms using Google and Google Scholar. Following the de-duplication of sources, the primary author conducted a title screen to eliminate irrelevant articles, identifying titles with phrases such as “atypical presentation” and “a rare case report.” We used the Covidence systematic review management system for reference management and screening at the abstract level.¹⁸ Our study was registered as a scoping review with the Center for Open Science.¹⁹

Three reviewers evaluated the abstracts using pre-defined inclusion and exclusion criteria. Inclusion criteria required that the study evaluated NREs or events similar to NREs, such as potential adverse events, near-misses, interruptions, and disturbances. The studies were included if performed in a high-risk setting with a methodology involving video review or real-time observation of events. The latter requirement limited the search to articles with objective ascertainment of events, a feature absent from articles relying only on voluntary reporting. We only included articles that described features of the events so we could perform a post hoc classification of the events.

During the abstract review stage, we excluded studies fitting the following criteria: performance in a low-risk setting, case reports, inclusion of events only related to medication administration or another single category of team action, retrospective database reviews, or use of voluntary reporting instead of direct observation or retrospective video review. We excluded papers that only included a single category of team action because their inclusion would limit comparison to other papers that attempted to identify all types of events.

Data Extraction

After reaching a consensus on studies for the next stage of review, three reviewers independently extracted data from the articles (Appendix 2). Extracted data from each study included the clinical setting, number of observers used to determine NREs, level of training or profession of these observers, method of case review (e.g., video or real-time observation), number of cases analyzed, number of NREs identified, and method used to classify NREs (Table 1). To evaluate the classification methods of NREs, two reviewers independently used the JCAHO patient safety event taxonomy to assign one or more classifications (impact, type, domain, cause, and prevention and mitigation) to the events described by each study.¹⁵ “Impact” classification was assigned to articles that assessed outcomes or effects of the NREs, typically reported as harm to the patient or delay in care.^{5,12,14,16,20–31} “Type” classification was assigned to articles that classified NREs by the underlying mechanism by which the event was affected, such as communication or teamwork, or by a specific error type, such as omission or commission.^{5,8,12,14,16,17,20,21,23,28,32–34} The “domain” of the articles included in the review was pre-determined at the time of article selection as “high-risk” but was further defined when the analysis was restricted to specific parts or phases of the setting in which the NREs occurred.^{5,8,13,22,28,30,35,36} “Cause” classification was assigned to articles that identified and reported contributing factors or agents (e.g., patient or practitioner) that led to NREs.^{5,8,12,14,16,17,21–31,33–35,36} Finally, the “prevention and mitigation” classification was assigned to the articles that evaluated measures (e.g., checklists) to reduce NREs.^8,14,29,36

Table 1.

Classification Method of Studies Evaluating Non-routine Events in High-Acuity Settings

Authors	Method	Case Type	Case Specialty	Sample Size/Type	JCAHOa Classification
Healey, AN et al. (2007)20	real-time	operative	urology	30 cases	impact, type
Wiegmann DA, et al. (2007)21	real-time	operative	cardiac surgery	31 cases	impact, type, cause
Barach, P, et al. (2008)22	real-time	operative	cardiac surgery, general surgery	102 cases	impact, domain, cause
Burtscher, MJ, et al. (2010)8	video	operative	anesthesiology	22 cases	type, domain, cause, prevention/mitigation
Gelbart, B, et al. (2010)35	video	critical care	obstetrics, neonatology	20 cases	domain, cause
McGillis Hall, L, et al. (2010)23	real-time	critical care	pediatrics	384 hours	impact, type, cause
Parker, SE, et al. (2010)24	real-time	operative	cardiac surgery	12 cases	impact, domain, cause
Savoldelli, GL, et al. (2010)25	video	operative	anesthesiology	37 cases	impact, cause
Williams, AL, et al. (2010)32	video	critical care	neonatology	12 cases	type, domain
Forster, AJ, et al. (2011)26	real-time	operative, critical care	cardiac surgery, internal medicine, obstetrics	1406 patients	impact, cause
Kahol, K, et al. (2011)16	real-time	trauma	trauma	10 cases	impact, type, cause
Pereira, BMT, et al. (2011)37	real-time	operative	trauma surgery	50 cases	cause
Schraagen, JM, et al. (2011)12	real-time	operative	cardiac surgery	40 cases	impact, type, cause
Campbell, G, et al. (2012)27	real-time	operative	anesthesiology	30 cases	impact, cause
Hu, YY, et al. (2012)14	video	operative	general surgery, surgical oncology	10 cases	impact, type, cause, prevention/mitigation
Bellora, E, et al. (2013)36	real-time	operative	general surgery, orthopedic surgery	61 cases	domain, prevention/mitigation
Dedhia, RC, et al. (2013)28	real-time	operative	otolaryngology	23 cases	impact, type, domain, cause
Yamada, NK, et al. (2015)17	video	critical care	neonatology	23 cases	type, cause
Raman, J, et al. (2016)29	real-time	operative	cardiac surgery	380 cases	impact, cause, prevention/mitigation
Webman, RB, et al. (2016)5	video	trauma	trauma	39 cases	impact, type, domain, cause
Blikkendaal, MD, et al. (2017)30	video	operative	gynecology	40 cases	impact, domain, cause
Cohen, TN, et al. (2017)33	real-time	operative	cardiac surgery	study I- 15 cases study II- 25 cases	type, cause
Slagle, JM, et al. (2018)13	video	operative	anesthesiology	319 cases	type, domain
Hamilton, EC, et al. (2018)34	real-time	operative	pediatric surgery	211 cases	type, cause
Law, KE, et al. (2018)31	real-time	operative	gynecology	45 cases	impact, cause

^aJCAHO- joint commission on accreditation of healthcare organizations

RESULTS

Search Results

The initial search yielded 9,089 articles after de-duplication. Following a title screen to eliminate irrelevant articles and case reports, 2,407 articles underwent abstract review, and 134 underwent full-text review. Among those undergoing full review, 68 were excluded due to methodology (e.g., retrospective chart review, database study), 36 due to no description of NREs (e.g., focus on medication errors, leadership behaviors), and five due to the setting (e.g., general medical ward, outpatient clinic) (Figure 1). We included 25 full-text articles meeting inclusion criteria in the final review (Table 1).

Figure 1.

PRISMA Diagram Showing Selection of Articles for Review

Description of Included Studies

Sixteen studies used real-time, in-person prospective surveillance of events, while the other nine studies used retrospective video review. Five studies used the term “NRE” to describe events, while the others used language such as “potential adverse events,” “distractions,” and “interruptions” to describe NREs.^5,8,12,13,31 The most common setting was the OR (n=19), followed by the ICU (n=5). One study included observations of adverse and potential adverse events in both the OR and ICU.²⁶ Two papers included an analysis of NREs occurring in the setting of trauma resuscitation in the ED.^5,16 Studies that focused on NREs in the OR included procedures performed in a range of specialty areas, including urology (n=1), cardiac surgery (n=7), general surgery (n=3), trauma surgery (n=1), surgical oncology (n=1), orthopedic surgery (n=1), otolaryngology (n=1), gynecology (n=2), and pediatric surgery (n=1).^{12,14,20–22,24,26,28–31,33,34,36,37} Four papers with a focus on NREs in the OR included only those related to anesthesiology providers.^8,13,25,27

Twenty-four studies included information about the observers documenting the NREs. In 13 studies, observers had a medical background, including nurses, medical students, residents, fellows, or attending physicians who were familiar with or worked in the setting.^{5,8,13,14,20,21,24,25,27,28,32,34,,35} Four studies included only observers without medical experience.^12,23,31,33 Fourteen studies included more than one observer.^{5,14,22–25,28,30–35,37} Interrater reliability testing was reported in seven studies.^{23,24,30–,34}

Among the 18 studies conducted only in an OR setting, a range of 10 to 380 cases were observed per study (median=40 cases).^{8,12–14,20–22,24,25,27–31,33,34,36,37} One study included NREs occurring in both the OR and ICU over an aggregate observation period of 1,406 hours.²⁶ In studies performed in an ICU setting, one included an evaluation of 384 hours of observations in a pediatric critical care setting, while three studies included an analysis of NREs occurring during medical resuscitation (range 12–250 resuscitations).^17,23,32,35 Two studies included an analysis of NREs occurring during trauma resuscitation (range 10–39 resuscitations).^5,16 Although NREs were observed in all studies, not every case or patient-hour had an NRE. The findings of each study showed that NREs are common but there was a wide variation in the number of NREs identified per case. The frequency of NREs per case in the included studies ranged from 30 NREs occurring in 380 cases (average 0.08 NREs per case) to 4,233 NREs in 25 cases (average 169.3 NREs per case).^29,33

Distribution of JCAHO Taxonomy Classifications among Included Studies¹⁵

Twenty-one studies included a classification of NREs by “cause,” attributing events to internal and external systems failures, and identifying components such as equipment availability, patient disease, and the provider associated with the NRE. The provider roles were pre-defined based on who was expected to be present in each clinical setting.^{5,8,12,14,16,17,21–31,33–35,37} Fourteen studies included a “type” classification of the involved process causing the NRE or classified based on an overarching theme.^{5,13,16,17,23,28,34} In studies that classified NREs by “type,” NREs were frequently attributed to failures in team communication or patient management.^{5,8,12,14,20,21,28,32,33}

Sixteen studies reported NREs by “impact” by assessing the time expense of the NRE or the potential for harm. Studies that included an analysis of the time cost of NREs used scales ranging from no temporal disruption to full case cessation.^{14,20,21,28–31} Studies that classified NREs based on the potential for harm included several definitions of harm categories, including minor/major, no harm/harm, and positive/negative.^{5,16,22,23,25–27} A common conclusion was that events propagating to a harmful outcome were rare, but mitigated events that could cause harm were common.^12,22,24,26

The “domain” category encompasses the settings in which events occur and was selected for by our inclusion criteria of “high-risk” clinical settings. Ten studies created more narrow domains by classifying NREs based on the phase of the event in which each occurred. The studies of NREs in the OR reported the occurrence of NREs in distinct periods such as “medication phase” and “intubation phase” or “induction,” “maintenance,” and “emergence.”^{8,13,22,24,28,30,36} Studies conducted in the ICU or in a trauma resuscitation setting included analyses within phases defined by the protocols used in these domains, e.g., Pediatric Advanced Life Support and Advanced Trauma Life Support.^5,32,35 Four studies that used “prevention and mitigation” included an assessment of the utility of a checklist or reporting system as a method for preventing the occurrence of NREs. “Prevention and mitigation” was also classified by the type of compensatory method used to mitigate the impact of NREs.^8,14,29,36

Most studies (n=24) included NRE analysis that involved more than one JCAHO classification method.¹⁵ Although no study used all five JCAHO patient safety classifications, four studies used four classes: one study included analysis by type, domain, cause, and prevention and mitigation; one by impact, type, cause, and prevention and mitigation; and two by impact, type, domain, and cause (Appendix 2).^5,8,14,28

DISCUSSION

This scoping review presents the findings of the medical NRE analyses performed in high-risk clinical domains over the past decade. Because of the diversity of settings, differences in the patient care processes, and different terminology used in each study, comparison of NREs and their features between studies is difficult. The development of a unified definition of NREs and their classifications will facilitate this type of comparison and allow for NRE analyses in aggregate across clinical domains. The main purpose of this review was to determine the optimal approach for NRE ascertainment and description that would aid in assessment of the burden of NREs on healthcare. We applied the JCAHO patient safety taxonomy to describing these events, an accepted nomenclature that has been used in studies of adverse events and patient safety.^38–40 A benefit of the JCAHO taxonomy is that outlined classifications are interrelated. For example, the “domain” creates an environment in which “causes” can lead to different “types” of NREs that have varying “impacts” and require different strategies for “prevention and mitigation.” We propose that this or a similar method of classifying NREs is needed when evaluating NREs in high-risk medical settings. The use of the common language from the JCAHO taxonomy will enable a future meta-analysis of the overall burden of NREs in these settings.¹⁵

Structuring an NRE Analysis

The studies included in the review used several methods for analyzing NREs. A combination of methods from different studies could yield the optimal NRE assessment approach. Real-time, in-person observation is known to cause behavior and practice alterations (i.e., Hawthorne effect) because providers know they are being observed.^{13,14,25,28–31} Analysis of video recordings is an alternative that allows for repeated reviews from multiple angles.^30,35 Video review may reduce the potential for a Hawthorne effect when compared to in-person observation.^{13,14,25,28–31,41,42} Despite the potential advantages of video review, this method may not be an option because of the associated costs of recording systems, the legal implications of recording patient care, or the use of a fixed camera position limiting a complete view of the clinical scene.^{8,17,30,35,43} Few studies that have directly compared these two methods of review have been performed in simulation and not in actual clinical practice.^44,45 Both video review and real-time observation can be considered appropriate methods for evaluating NREs until direct comparison has been performed in practice.

Understanding the clinical contexts associated with NREs reduces the risk of misinterpreting why an event occurred when using only retrospective case review.⁴⁶ Because voluntary reporting systems may lead to underreporting of NREs due to lack of incentive to report all events, we excluded studies if direct observation was not used.^2,3,47 Studies that use a combination of voluntary reporting and facilitated surveys have identified workflow deviations that can lead to patient harms but retain the bias of self-report.^3,4 Direct observation supplemented by voluntary reporting, facilitated interview, and chart review may improve NRE ascertainment and should be considered in future studies. The use of other modes of NRE reporting will increase the validity of NRE review.

The use of observers with medical expertise to identify NREs was included in only 13 studies. Observers with domain experience may identify nuances that would not be noticeable to others. Expert review, however, may be associated with the higher cost and may not be feasible in specialized domains where the availability of individuals for performing these time-consuming studies is limited.^5,32 The familiarity of experts with the clinical setting also may lead to bias and loss of objectivity. The bias can be mitigated with the use of multiple observers and the measurement of interrater reliability, a method used in several studies.^{23,24,30–34} The use of multiple observers and assessment of interrater reliability should be included to support reproducibility.

NREs as Safety Concerns

Studies in this scoping review showed that NREs are frequent in high-risk medical settings, but the difference in NRE language lead to a wide variation in the number of NREs identified in each study. A common conclusion was that the identified NREs have potential impacts on patient safety, care delivery, and overall outcomes. Although most NREs did not cause direct patient harm, NREs were considered precursor events or latent safety threats in each study. From this perspective, understanding the factors that contribute to NREs will aid in the development of strategies for mitigating or preventing patient harm.^{5,8,13,21,23,24,26,31,32} The underlying cause of NREs was often multifactorial, relating to the involved providers, the clinical status of the patient, or the complexity of the patient care task.^{5,12–14,16,17,20–,22,25,27,28,30,31,33,37} To address provider-related factors related to NREs, several strategies have been proposed to improve teamwork and communication, such as perioperative briefings, closed-loop communication, effective decision making, and increased information sharing.^{8,12,14,21,28,31,32} Other systems, such as revisions to pre-existing hospital protocols, can address problems originating external to the team that create preconditions for unsafe acts. Adjuncts such as checklists and timeout pauses also have been suggested and successfully used as interventions to reduce NREs and adverse outcomes.^5,8,17,29,36

Limitations

This study has several limitations. The search strategy included three relevant databases but had a year restriction placed on publications. Although the practice of NRE identification is not novel, the application of NRE classifications in healthcare has been a relatively new focus, supporting this restriction.^1,6 Only publications in English were reviewed. Additional reports of studies in other languages may exist that were not included. Although we included generic terms in our search, our search may have missed papers because of the lack of standardized terminology for describing NREs. Our review highlighted the importance of standardizing terminology for future reviews to address this limitation. Although we identified common themes and proposed future areas of study, our review included only 25 papers. Because of the limited number of available studies, we were only able to aggregate studies at a high-level. As more work is published in this area, additional insights into the optimal approach for ascertaining, classifying, and establishing the relevance of NREs will emerge.

Conclusions

Our review of studies addressing NREs in high-risk medical settings shows that NREs are common but rarely contribute directly to patient harm. The authors of several studies have proposed that NREs may represent latent safety threats to patients. A comparison of the results of studies of NREs will be facilitated by the use of shared definitions and classifications. Our proposed use of the taxonomy outlined by JCAHO may allow an aggregate analysis of the health burden of NREs in high-risk domains.