Understanding the limitations of incident reporting in medication errors

Editor—We read with interest the recent study published in the British Journal of Anaesthesia reporting an analysis of 1970 medication-related adverse events in anaesthesia over a 10-yr period.¹ A range of similar studies have been published over the last 20 yr that have explored a vast number of harms and near misses across multiple clinical contexts. These have attempted to collate the frequency, range, types, causes, and outcomes with the aim of ‘finding and fixing’ the sources of clinical mishaps. The study by Sanduende-Otero and colleagues¹ is unusual in reporting a range of events outside the USA and UK, and the corrective actions that were taken as a consequence. However, prior analysis and critique of these types of data still stand.² In thinking about the limitations of this and other similar studies, we felt it is worth considering how science might need to move forward.

It is well acknowledged, including in this paper, that incident reporting under-represents the frequency of events, possibly by a factor of 20.³ What is less well acknowledged is that certain types of events are more likely to be reported than others, and within each type of event, different levels of risk and harm yield different reporting frequencies.4, 5, 6 This is partly a consequence of the very well-acknowledged effects of hindsight and outcome bias,⁷ but also extends to the ability to discern ‘right’ from ‘wrong’, which is far easier when considering administration (a heavily proceduralised activity, where deviations before harm are observable) than prescription (cognitive, unobservable, and only identifiable as problematic in retrospect). One particular criticism of the study by Sanduende-Otero and colleagues¹ is that a denominator is not applied to their data. In other words, the relative frequencies of incidents, causes, and the observed harm are at least as much a function of the likelihood of someone to report them as they are a function of their actual frequency. An inability to distinguish between the two leaves these data largely unhelpful and potentially misleading. It also skews data towards clear and recognised risks and less sophisticated or comprehensive analyses, rather than telling us much about what is actually going on.

Analysis is also a function of what is reported and the limitations of the classification systems, rather than a true reflection of what happened in each case.⁸ Not all clinicians are equally aware of just how tasks, technologies, workspace, and organisation interact to influence their performance,⁹ so details are often omitted from the reports themselves. Moreover, causality is neither linear nor deterministic, so classification systems that require attribution of complex, interacting, multifactorial events to a single causal factor are by their nature simplistic and misrepresentative.¹⁰ Thus, the attribution of these events offers little insight into the causes, and even less as to the solutions. This is not to say that there is no value in these reports and data, but rather we need to extend our understanding and assessment of these events and fully recognise their deficits.

Although it is a strength of the paper that the corrective actions used were detailed, they demonstrate a limited perspective on performance and process improvement, which largely seeks ‘human error’ as the problem, and prioritizes behaviour change solutions. This suggests a lack of understanding of effective system redesign in managers and senior clinicians, and is held in contrast to a systems approach advocated by safety scientists that would address the complex adaptive interactions between tasks, technologies, the working environment, and the organisation that are required for success in everyday clinical work.¹¹ We recognise that correcting some, if not a large portion, of errors will require changes in infrastructure at an institutional or industry level. However, we cannot and should not ignore large classes of hazards that will require the pharmaceutical industry to adapt (lookalike, soundalike, or different medications packaged in similar coloured and shaped containers) or device manufacturers to innovate and change current standards (working together to manufacture and adopt new connectors for epidural tubing), as in barcode scanning to reduce syringe swap errors.¹² Moreover, a ‘find and fix’ approach may be more limited than the resilience engineering perspective¹³ that also seeks to identify and promote successful work contexts.

In looking for ‘errors’ rather than harms, and with the field dominated by clinicians, not safety scientists, healthcare in general has failed to embrace the socio-technical complexity of adverse events.¹⁴ Progress in human factors in surgical and anaesthetic safety will help to move beyond checklists, teamwork, counting ‘errors’, and the limited view of human fallibility co-opted by the clinical profession, and instead facilitate collaboration with human factors experts and other safety scientists to understand and create working environments and clinical systems, where clinicians can perform to the best of their ability.¹⁵

We need to be realistic about the uses and limitations of incident reports, move beyond outdated views of systems safety, and work with safety scientists to adopt appropriately nuanced views of harm avoidance and solutions. When publishing analysis of incident reports, there should be clear consideration not just of the incidents, but a thorough consideration of the limitations of these data, critique of the systems in which they were collected, consideration of the science that underlies them (such as methods to address hindsight and outcome bias), and suggestions for other complimentary approaches. Perhaps, then, we (clinicians and safety scientists together) stand a chance of really making a difference to our patients over the next decade.

Declarations of interest

The authors declare that they have no conflicts of interest.