Accidental injustice: Healthcare AI legal responsibility must be prospectively planned prior to its adoption

Abstract

This article contributes to the ongoing debate about legal liability and responsibility for patient harm in scenarios where artificial intelligence (AI) is used in healthcare.We note that due to the structure of negligence liability in England and Wales, it is likely that clinicians would be held solely negligent for patient harms arising from software defects, even though AI algorithms will share the decision-making space with clinicians.Drawing on previous research, we argue that the traditional model of negligence liability for clinical malpractice cannot be relied upon to offer justice for clinicians and patients. There is a pressing need for law reform to consider the use of risk pooling, alongside detailed professional guidance for the use of AI in healthcare spaces.

The difficulty with AI implementation is that we live in a world where humans have historically been the actors. We had always been the ones who had wielded the paintbrush, driven the car, and made the complex calculations. The earnest adoption of technology as it has arisen has yielded incredible efficiencies when compared to human efforts alone in many fields (see Hidden figures¹ for an example of this in action in 1960s space engineering). But what the technology does practically is less of an issue compared to the effect of using the technology, which raises concerns that might prevent immediate and widespread adoption.

Medicine and technology are fields of expertise which are dependent on cultural acceptance for their success. Suspicion of the ‘new’ is a common theme throughout the histories of medicine and technology; for instance, the contributions of Hippocrates to medicine were a significant departure from the supernatural tradition which preceded him. Luddite riots in the 19th century showed that insensitive implementation of automation can lead to social rejection of technology.

There is an oft-repeated aphorism, attributed to the computer scientist John McCarthy, that ‘as soon as it works, no one calls it AI anymore’.² For AI to go broadly unchallenged in healthcare contexts, developers of healthcare AI applications must demonstrate that their software works as intended and is an improvement on existing, human-led strategies. The corpus of clinical knowledge has been developed over hundreds of years; refined and packaged up into healthcare services, constituting thousands of professional and institutional moving parts. So, the bar is set high.

Healthcare standards are exacting for good reason. With the growth of healthcare as a professional endeavour has come the development of case law determining how society responds when something goes wrong, eg in the case of negligence on the part of the healthcare professional. But the introduction of AI to inform the clinician's thought processes is fundamentally altering this as its adoption introduces new actors (the AI and those who created it) into the decision-making space.

While the aim is that AI will improve healthcare outcomes, we must cautiously accept that not all AIs (or even healthcare professionals) are perfect; an AI might output and a clinician could subsequently use recommendations which are not suitable for the patient in question, and that there is a potential risk for patient harm to eventuate due to such human and computing flaws.

Imagine that a clinician uses an AI in their practice and, because of using the AI's output, a patient comes to harm. Who shoulders the legal responsibility for this? We've investigated how tort law might evolve in this scenario and found that it may very well be the clinical user who shoulders the burden of liability for AI use.^3,4 Our analysis found that, even though the AI shares the decision-making space with the clinician, it's likely that the clinician would be held negligent as they would have been the actor most ‘proximate’ (in a legal sense) to the risk of harm.³ We don't think that's on. At all.

This situation would be unfair as the clinician would not have made the decision alone in this scenario; the AI application would also have influenced what happened in that patient's care. There is no such thing as a perfect computer system. Because it can be foreseen that an AI might make an error, then – if a patient is harmed – those involved in creating the AI can also be held responsible for their involvement in the harm(s) done, alongside the clinical user.⁵ Indeed, it is unfair and inefficient to disconnect accountability from the locus of control of the information that is presented to the clinician.⁶ To account for the different contributions made by users and AI developers, it has been argued that a shared model of responsibility between clinicians and AI developers would be ethically fairer.⁵

So, is negligence the best way to regulate the appropriate use of AI in healthcare contexts? In short, our analysis suggests that negligence liability taken alone is unable to bear the regulatory demands placed upon it.³ In the traditional view, where the clinician provides care to their patient in the absence of artificial intelligence, negligence liability balances the need to protect patient safety while respecting the need for clinicians to apply their expertise and exercise appropriate professional judgement. It does this by providing a deterrent against unduly risky behaviour, as well as providing a route for compensation and restitution where patient safety has been compromised.

The common law expands the scope of negligence liability incrementally. However, judges cannot radically modify the traditional criteria for negligence liability to respond to the challenges of regulating AI, meaning there remains the potential for ‘crushing’ liability to fall squarely on the shoulders of clinicians. For AI developers, the innovation risk would be rewarded with commercial success if they brought a safe and effective AI solution to market. But they currently do not share in the risk of failure. As such, we recognise the potential for clinicians to be treated as moral and legal crumple zones⁷ positioned to absorb responsibility which, ethically, should not be solely theirs to bear. Without the threat of liability falling on AI developers, the incentives to encourage responsible innovation, which prioritises patient safety above all else, are reduced.

We have previously argued for reforms to the law of negligence to account for risk pooling.³ This should ensure that collaborations between clinicians and AI developers enhance patient protection and contribute to the public good.⁸ Risk pooling recognises that in complex endeavours, such as in providing care to patients, multiple parties may have contributed significantly to the outcome. Essentially, users and developers would share the risk of AI use. Without reform, a lack of equitable sharing of responsibility (such as risk pooling) could lead to widespread injustice.

However, risk pooling cannot be developed in common law by the courts. Instead, legislation is required to modify the law to implement it. Examples of such schemes include joint liability for mesothelioma, which was enacted in the Compensation Act 2006 s3, and the ‘insurer-first’ liability scheme introduced by the Automated and Electric Vehicles Act 2018 s2.

Meanwhile, AI is already in use in a growing number of medical and healthcare contexts. Our findings lead us to conclude that the need for legislative and regulatory intervention is increasingly urgent. In the absence of new legislation, however, clinicians at least need professional guidance to give them the confidence that they will be unlikely to breach their duty of care to patients if the appropriate clinical standards are followed.^9,10

Ethics Information

Not applicable.

CRediT authorship contribution statement

Kit Fotheringham: Writing – review & editing, Writing – original draft, Conceptualization. Helen Smith: Writing – review & editing, Writing – original draft, Conceptualization.

Declaration of competing interest

Helen Smith reports financial support was provided by Wellcome Trust. Other authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.