Error in radiology—where are we now?

Abstract

Error is inherent in radiological practice. Our awareness of the extent of this and the reasons behind it has increased in recent times. Our next step must be the development of a shared understanding with our patients of the limitations as well as the huge benefits of medical imaging.

In an important article published in this journal in 1997, Robinson wrote of error in image interpretation as radiology’s “Achilles’ heel”.¹ In a wide-ranging review he discussed definitions of error and variation as well as strategies for their reduction, and looked forward to a future in which computer analysis could potentially replace not only the perception function of the human observer but also the function of interpretation. Two decades on, what progress have we made?.

Firstly it is safe to say that the problem has not gone away. Evidence gathered during the plain film era suggested a radiologist error rate of around 3–5% in daily practice.² We also know that rates of interpretative error in cross-sectional imaging are significantly higher, of the order of 20–30%.³ In one study, expert abdominal radiologists recorded major discrepancy rates of 26% when reviewing each others’ work and up to 32% between their own interpretations of the same CT images on different occasions.⁴ As more and more of the time of radiologists is now taken up with reporting cross-sectional imaging studies, it is reasonable to assume that our rate of error has correspondingly increased.

Moreover, as the use of CT and MRI grows, and as digital storage means that previous images are always available for comparison, opportunities to look back at previous studies and discover errors in hindsight have multiplied. This trend will of course continue as the number of occasions on which there are previous studies to compare increases – the more we scan, the more material there will be for retrospective review in the future.

Our awareness of error in radiological practice has accordingly increased over recent decades.⁵ We have moved on from believing that errors were only committed by trainee, junior or inexpert radiologists to an understanding that the susceptibility to error afflicts us all. Even the most expert radiologists have had to come to terms with the fact that retrospective review of previous imaging will sometimes reveal abnormalities which, however subtle, they will feel that they could have detected or interpreted in a different way.

Robinson correctly predicted that digital image acquisition and display would not of themselves affect observational error rates but new technologies have introduced new sources of error. The widespread adoption of speech recognition technology for reporting has seen the replacement of traditional “typographical errors” with a new generation of errors ranging from the mildly amusing to the frankly dangerous.⁶

Perceptive and cognitive biases

Our understanding of the reasons behind our errors has also increased. As Robinson wryly observed, the performance of the human eye and brain has failed to keep pace with the technical progress of radiology. Simple optical illusions demonstrate how the human eye can be deceived and radiologists are as susceptible as anyone else.⁷ We have to learn, for example, that juxtaposition of regions of different attenuation on a cranial CT scan can distort our perception, rendering us more or less likely to make a correct diagnosis of haemorrhage. Similarly our visual appraisal of the size of a lung lesion on sequential imaging must often be verified by direct measurement in order to avoid making an incorrect assessment of growth or shrinkage.

Radiology involves a process of making decisions under conditions of uncertainty. The relatively recent popularisation of the work of psychologists Daniel Kahnemann and Amos Tversky as well as a subsequent generation of behavioural economists has greatly increased our understanding of how such decisions are made.⁸ Specifically, the dual process theory of human cognition explains how much of our decision-making is achieved through the use of short cuts or heuristics rather than through a deliberative process of evaluation, and how this can lead us into predictable patterns of error. Experiments conducted in a wide range of different circumstances have identified a series of cognitive biases which can influence to a much greater degree than we are aware our interpretation of a particular image in our daily practice.⁹ Some of these biases such as “satisfaction of search” are familiar to radiologists, although still difficult to overcome. Others such as priming and framing, whereby factors such as the clinical information provided, the context in which we are reporting or even the conversation going on elsewhere in the reporting room can affect our reports are less well-known. Confirmation bias, by which a rapid impression is formed and the evidence then weighted to support that initial impression, is a recurring danger for all radiologists.

There is a risk that an increased understanding of the inevitability of some error in our practice might lead to a degree of complacency. To date, evidence of the effectiveness of “de-biasing” is lacking, but an awareness of our individual and collective cognitive blind spots can and should help us to develop strategies to address their impact on our work. There is ample scope to address deficiencies in our systems of work, including ergonomical and environmental factors, display protocols and the recognition and management of fatigue.

The role of computers

The pace of development of computer analysis of medical images has accelerated in the last few years. The application of artificial intelligence (AI) and particularly machine learning to radiology is still in its relative infancy but many of the advances foreseen by Robinson appear on the cusp of making a real clinical impact. In particular, the use of a computer as a “second reader” for many types of examination seems likely to bring significant benefits. Debate continues over the extent to which AI will replace radiologists at some point in the future. At the very least it seems reasonable for now to regard it as a tool which, like the advent of digital imaging, will result in a step-wise productivity gain, enabling an apparently inadequate number of radiologists to continue to cope with otherwise unsupportable increases in demand for imaging.

As with other new technologies mentioned above, it is also likely that the introduction of AI into radiology practice will bring new sources of error. Perhaps we will see an increase in “overcalls” as the sensitivity of the technology exceeds normal human perception. The vexed question of the extent to which the human is able or permitted to over ride the decision of the computer will also come into play as it has with other applications of AI such as driverless vehicles. As others have observed, accuracy in radiological interpretation can prove to be an illusory goal.¹⁰

Patients and the public

Unfortunately, the greater understanding of error–its frequency and to a certain extent its inevitability–which radiologists have gained over the past two decades is not shared even by colleagues in our own profession who often seem to have unrealistic expectations of the accuracy of radiological interpretation. Outside the profession, understanding is even less. One of the biggest problems facing radiologists now is the yawning gap between what we know to be our error rate and what our patients might believe it to be. The discovery in hindsight of an error in interpretation of a radiological image is now commonplace in our practice but is still often perceived by the patient as something shocking and exceptional, calling into question the competence of the radiologist involved. Addressing this disjunction must be one of the highest current priorities for radiologists and our professional bodies. It will not be easy. “Technical” solutions such as a process of consent prior to imaging or disclaimers on reports are unlikely to be popular or effective. Ultimately a process of public education is required which treads a fine line, explaining the pervasive nature of radiological error as well as the measures which we take to avoid it whilst emphasising the enormous benefit which radiology–despite its inherent flaws–continues to bring to patient care.